Firearm Training Systems and Methods

ABSTRACT

An imaging device captured images of a scene that includes at least one shooter. Each shooter of the at least one shooter operates an associated firearm to discharge one or more projectile. A positioning mechanism positions an infrared filter in and out of a path between the imaging device and the scene. A processing system processes images of the scene when the infrared filter is positioned in the path to detect projectile discharges in response to each shooter of the at least one shooter firing the associated firearm. The processing system processes images of the scene captured when the infrared filter is positioned out of the path to identify, for each detected projectile discharge, a shooter of the at least one shooter that is associated with the detected projectile discharge.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of U.S. patent applicationSer. No. 16/858,761, filed on Apr. 27, 2020, now U.S. Pat. No. ______,which is a continuation-in-part of U.S. patent application Ser. No.16/036,963, filed on Jul. 17, 2018, now U.S. Pat. No. 10,670,373, whichis a continuation of U.S. patent application Ser. No. 15/823,634, filedon Nov. 28, 2017, now U.S. Pat. No. 10,077,969. The disclosures of theaforementioned applications are incorporated by reference in theirentirety herein.

TECHNICAL FIELD

The present invention relates to firearm target training.

BACKGROUND OF THE INVENTION

Firearm target training systems are generally used to provide firearmweapons training to a user or trainee. Traditionally, the user isprovided with a firearm and discharges the firearm while aiming at atarget, in the form of a bullseye made from paper or plastic. Thesetypes of training environments provide little feedback to the user, inreal-time, as they require manual inspection of the bullseye to evaluateuser performance.

More advanced training systems include virtual training scenarios, andrely on modified firearms, such as laser-based firearms, to train lawenforcement officers and military personnel. Such training systems lackmodularity and require significant infrastructural planning in order tomaintain training efficacy.

SUMMARY OF THE INVENTION

The present invention is a system and corresponding components forproviding functionality for firearm training.

According to the teachings of an embodiment of the present invention,there is provided a firearm training system. The firearm training systemcomprises: an imaging device deployed to capture images of a scene, thescene including at least one shooter, each shooter of the at least oneshooter operating an associated firearm to discharge one or moreprojectile; an infrared filter; a positioning mechanism operativelycoupled to the infrared filter, the positioning mechanism configured toposition the infrared filter in and out of a path between the imagingdevice and the scene; a control system operatively coupled to thepositioning mechanism and configured to: actuate the positioningmechanism to position the infrared filter in and out of the path, andactuate the imaging device to capture images of the scene when theinfrared filter is positioned in and out of the path; and a processingsystem configured to: process images of the scene captured when theinfrared filter is positioned in the path to detect projectiledischarges in response to each shooter of the at least one shooterfiring the associated firearm, and process images of the scene capturedwhen the infrared filter is positioned out of the path to identify, foreach detected projectile discharge, a shooter of the at least oneshooter that is associated with the detected projectile discharge.

Optionally, the at least one shooter includes a plurality of shooters,and each shooter operates the associated firearm with a goal to strike atarget with the discharged projectile, and the firearm training systemfurther comprises: an end unit comprising an imaging device deployed forcapturing images of the target, and the processing system is furtherconfigured to: process images of the target captured by imaging deviceof the end unit to detect projectile strikes on the target, andcorrelate the detected projectile strikes on the target with thedetected projectile discharges to identify, for each detected projectilestrike on the target, a correspondingly fired firearm associated withthe identified shooter.

Optionally, the target is a physical target.

Optionally, the target is a virtual target.

Optionally, the positioning mechanism includes a mechanical actuator inmechanical driving relationship with the infrared filter.

Optionally, the positioning mechanism generates circular-to-linearmotion for moving the infrared filter in and out of the path from thescene to the imaging device.

Optionally, the imaging device includes an image sensor and at least onelens defining an optical path from the scene to the image sensor.

Optionally, the firearm training system further comprises: a guidingarrangement in operative cooperation with the infrared filter anddefining a guide path along which the infrared filter is configured tomove, such that the infrared filter is guided along the guide path andpasses in front of the at least one lens so as to be positioned in theoptical path when the positioning mechanism is actuated by the controlsystem.

Optionally, the projectiles are live ammunition projectiles.

Optionally, the projectiles are light beams emitted by a light sourceemanating from the firearm.

Optionally, the control system and the processing system are implementedusing a single processing system.

Optionally, the processing system is deployed as part of a serverremotely located from the imaging device and in communication with theimaging device via a network.

There is also provided according to an embodiment of the teachings ofthe present invention a firearm training system. The firearm trainingsystem comprises: a shooter-side sensor arrangement including: a firstimage sensor deployed for capturing infrared images of a scene, thescene including at least one shooter, each shooter of the at least oneshooter operating an associated firearm to discharge one or moreprojectile, and a second image sensor deployed for capturing visiblelight images of the scene; and a processing system configured to:process infrared images of the scene captured by the first image sensorto detect projectile discharges in response to each shooter of the atleast one shooter firing the associated firearm, and process visiblelight images of the scene captured by the second image sensor toidentify, for each detected projectile discharge, a shooter of the atleast one shooter that is associated with the detected projectiledischarge.

Optionally, the at least one shooter includes a plurality of shooters,and each shooter operates the associated firearm with a goal to strike atarget with the discharged projectile, and the firearm training systemfurther comprises: an end unit comprising an imaging device deployed forcapturing images of the target, and the processing system is furtherconfigured to: process images of the target captured by imaging deviceof the end unit to detect projectile strikes on the target, andcorrelate the detected projectile strikes on the target with thedetected projectile discharges to identify, for each detected projectilestrike on the target, a correspondingly fired firearm associated withthe identified shooter.

Optionally, the target is a physical target.

Optionally, the target is a virtual target.

There is also provided according to an embodiment of the teachings ofthe present invention a firearm training method. The firearm trainingmethod comprises: capturing, by at least one image sensor, visible lightimages and infrared images of a scene that includes at least oneshooter, each shooter of the at least one shooter operating anassociated firearm to discharge one or more projectile; and analyzing,by at least one processor, the captured infrared images to detectprojectile discharges in response to each shooter of the at least oneshooter firing the associated firearm, and analyzing, by the at leastone processor, the captured visible light images to identify, for eachdetected projectile discharge, a shooter of the at least one shooterthat is associated with the detected projectile discharge.

Optionally, the at least one image sensor includes exactly one imagesensor, and infrared images are captured by the image sensor when aninfrared filter is deployed a path between the image sensor and thescene, and the visible light images are captured by the image sensorwhen the infrared filter is positioned out of the path between the imagesensor and the scene.

Optionally, the at least one image sensor includes: an infrared imagesensor deployed for capturing the infrared images of the scene, and avisible light image sensor deployed for capturing the visible lightimages of the scene.

Optionally, the at least one shooter includes a plurality of shooters,and each shooter operates the associated firearm with a goal to strike atarget with the discharged projectile, and the firearm training methodfurther comprises: capturing, by an imaging device, images of thetarget; analyzing, by the at least one processor, images of the targetcaptured by imaging device to detect projectile strikes on the target;and correlating the detected projectile strikes on the target with thedetected projectile discharges to identify, for each detected projectilestrike on the target, a correspondingly fired firearm associated withthe identified shooter.

Unless otherwise defined herein, all technical and/or scientific termsused herein have the same meaning as commonly understood by one ofordinary skill in the art to which the invention pertains. Althoughmethods and materials similar or equivalent to those described hereinmay be used in the practice or testing of embodiments of the invention,exemplary methods and/or materials are described below. In case ofconflict, the patent specification, including definitions, will control.In addition, the materials, methods, and examples are illustrative onlyand are not intended to be necessarily limiting.

BRIEF DESCRIPTION OF THE DRAWINGS

Some embodiments of the present invention are herein described, by wayof example only, with reference to the accompanying drawings. Withspecific reference to the drawings in detail, it is stressed that theparticulars shown are by way of example and for purposes of illustrativediscussion of embodiments of the invention. In this regard, thedescription taken with the drawings makes apparent to those skilled inthe art how embodiments of the invention may be practiced.

Attention is now directed to the drawings, where like reference numeralsor characters indicate corresponding or like components. In thedrawings:

FIG. 1 is a diagram illustrating an environment in which a systemaccording to an embodiment of the invention is deployed, the systemincluding an end unit, a processing subsystem and a control subsystem,all linked to a network;

FIG. 2 is a schematic side view illustrating the end unit of the systemdeployed against a target array including a single target fired upon bya firearm, according to an embodiment of the invention;

FIG. 3 is a block diagram of the components of the end unit, accordingto an embodiment of the invention;

FIG. 4 is a schematic front view illustrating a target mounted to atarget holder having a bar code deployed thereon, according to anembodiment of the invention;

FIGS. 5A and 5B are schematic front views of a target positionedrelative to the field of view of an imaging sensor of the end unit,according to an embodiment of the invention;

FIGS. 6A-6E are schematic front views of a series of images of a targetcaptured by the imaging device, according to an embodiment of theinvention;

FIG. 7 is a block diagram of the components of the processing subsystem,according to an embodiment of the invention;

FIG. 8 is a schematic side view illustrating a firearm implemented as alaser-based firearm, according to an embodiment of the invention;

FIG. 9 is a block diagram of peripheral devices connected to the endunit, according to an embodiment of the invention;

FIG. 10 is a schematic front view illustrating a target array includingmultiple targets, according to an embodiment of the invention;

FIG. 11 is a diagram illustrating an environment in which a systemaccording to an embodiment of the invention is deployed, similar to FIG.1, the system including multiple end units, a processing subsystem and acontrol subsystem, all linked to a network;

FIG. 12 is a schematic representation of the control subsystemimplemented as a management application deployed on a mobilecommunication device showing the management application on a homescreen;

FIG. 13 is a schematic representation of the control subsystemimplemented as a management application deployed on a mobilecommunication device showing the management application on a detailsscreen;

FIG. 14 is a schematic side view similar to FIG. 2, and furtherillustrating an infrared filter (IR) assembly coupled to the end unit,according to an embodiment of the present invention;

FIG. 15A is a schematic front view illustrating an IR positioningmechanism and an IR filter of the IR filtering assembly, with the IRpositioning mechanism assuming a first state such that the IR filter ispositioned out of an optical path from a scene to an image sensor of theend unit;

FIG. 15B is a schematic front view illustrating the IR positioningmechanism and the IR filter, with the IR positioning mechanism assuminga second state such that the IR filter is positioned in the opticalpath;

FIG. 15C is a schematic front view illustrating the IR positioningmechanism and the IR filter, with the IR positioning mechanism assumingan intermediate state such that the IR filter is in transition from outof the optical path to into the optical path;

FIG. 15D is a schematic front view illustrating the IR positioningmechanism and the IR filter, with the IR positioning mechanism assuminganother intermediate state such that the IR filter is in transition fromin the optical path to out of the optical path;

FIG. 16A is a schematic side view corresponding to FIG. 15A;

FIG. 16B is a schematic side view corresponding to FIG. 15B;

FIG. 17 is a block diagram illustrating the linkage between the end unitand the IR filter assembly;

FIG. 18 is a block diagram of the components of an end unit having twoimage sensors that are separately used in different modes of operationof the system, according to an embodiment of the invention;

FIG. 19 is a schematic illustration of a system that supports jointfirearm training of shooters according to embodiments of the presentinvention, the system having a shooter-side sensor arrangement thatcaptures visible light and infrared images of shooters, as well as anend unit deployed against a target;

FIG. 20 is a schematic representation of a field-of-view (FOV)associated with the shooter-side sensor arrangement and sub-divided intomultiple regions, with a different shooter positioned in each region;

FIG. 21 is a block diagram of a processing unit associated with theshooter-side sensor arrangement, according to embodiments of the presentdisclosure;

FIG. 22A is a schematic front view illustrating an IR positioningmechanism and an IR filter of the IR filtering assembly, with the IRpositioning mechanism assuming a first state such that the IR filter ispositioned out of an optical path from a scene containing shooters to animage sensor of the shooter-side sensor arrangement;

FIG. 22B is a schematic front view illustrating the IR positioningmechanism and the IR filter, with the IR positioning mechanism assuminga second state such that the IR filter is positioned in the optical pathfrom the scene containing shooters to the image sensor of theshooter-side sensor arrangement;

FIG. 23A is a schematic side view corresponding to FIG. 22A;

FIG. 23B is a schematic side view corresponding to FIG. 22B; and

FIG. 24 is a block diagram of an imaging device, having a visible lightimage sensor and an infrared image sensor, for capturing visible lightand infrared images of a scene containing shooter, according toembodiments of the present disclosure.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention is a system and corresponding components forproviding functionality for firearm training.

Before explaining at least one embodiment of the invention in detail, itis to be understood that the invention is not necessarily limited in itsapplication to the details of construction and the arrangement of thecomponents and/or methods set forth in the following description and/orillustrated in the drawings and/or the examples. The invention iscapable of other embodiments or of being practiced or carried out invarious ways. Initially, throughout this document, references are madeto directions such as, for example, front and rear, top and bottom, leftand right, and the like. These directional references are exemplary onlyto illustrate the invention and embodiments thereof.

Referring now to the drawings, FIG. 1 shows an illustrative exampleenvironment in which embodiments of a system, generally designated 10,of the present disclosure may be performed over a network 150. Thenetwork 150 may be formed of one or more networks, including forexample, the Internet, cellular networks, wide area, public, and localnetworks.

With continued reference to FIG. 1, as well as FIGS. 2 and 3, the system10 provides a functionality for training (i.e., target training ortarget practice) of a firearm 20. Generally speaking, the system 10includes an end unit 100 which can be positioned proximate to a targetarray 30 that includes at least one target 34, a processing subsystem132 for processing and analyzing data related to the target 34 andprojectile strikes on the target 34, and a control subsystem 140 foroperating the end unit 100 and the processing subsystem 132, and forreceiving data from the end unit 100 and the processing subsystem 132.

With reference to FIG. 7, the processing subsystem 132 includes an imageprocessing engine 134 that includes a processor 136 coupled to a storagemedium 138 such as a memory or the like. The image processing engine 134is configured to implement image processing and computer visionalgorithms to identify changes in a scene based on images of the scenecaptured over an interval of time. The processor 136 can be any numberof computer processors, including, but not limited to, amicrocontroller, a microprocessor, an ASIC, a DSP, and a state machine.Such processors include, or may be in communication with computerreadable media, which stores program code or instruction sets that, whenexecuted by the processor, cause the processor to perform actions. Typesof computer readable media include, but are not limited to, electronic,optical, magnetic, or other storage or transmission devices capable ofproviding a processor with computer readable instructions. Theprocessing subsystem 132 also includes a control unit 139 for providingcontrol signals to the end unit 100 in order to actuate the end unit 100to perform actions, as will be discussed in further detail below.

The system 10 may be configured to operate with different types offirearms. In the non-limiting embodiment illustrated in FIG. 2, thefirearm 20 is implemented as a live ammunition firearm that shoots alive fire projectile 22 (i.e., a bullet) that follows a trajectory 24path from the firearm 20 to the target 34. In other embodiments, as willbe discussed in subsequent sections of the present disclosure, thefirearm 20 may be implemented as a light pulse-based firearm whichproduces one or more pulses of coherent light (e.g., laser light). Insuch embodiments, the laser pulse itself acts as the projectile.

In addition, the system 10 may be configured to operate with differenttypes of targets and target arrays. In the non-limiting embodimentillustrated in FIG. 2, the target 34 is implemented as a physical targetthat includes concentric rings 35 a-g. In other embodiments, as will bediscussed in subsequent sections of the present disclosure, the target34 may be implemented as a virtual target projected onto a screen orbackground by an image projector connected to the end unit 100. Notethat representation of the target 34 in FIG. 2 is exemplary only, andthe system 10 is operable with other types of targets, including, butnot limited to, human figure targets, calibration targets,three-dimensional targets, field targets, and the like.

As illustrated in FIG. 1, the processing subsystem 132 may be deployedas part of a server 130, which in certain embodiments may be implementedas a remote server, such as, for example, a cloud server or serversystem, that is linked to the network 150. The end unit 100, theprocessing subsystem 132, and the control subsystem 140 are all linked,either directly or indirectly, to the network 150, allowingnetwork-based data transfer between the end unit 100, the processingsubsystem 132, and the control subsystem 140.

The end unit 100 includes a processing unit 102 that includes at leastone processor 104 coupled to a storage medium 106 such as a memory orthe like. The processor 104 can be any number of computer processors,including, but not limited to, a microcontroller, a microprocessor, anASIC, a DSP, and a state machine. Such processors include, or may be incommunication with computer readable media, which stores program code orinstruction sets that, when executed by the processor, cause theprocessor to perform actions. Types of computer readable media include,but are not limited to, electronic, optical, magnetic, or other storageor transmission devices capable of providing a processor with computerreadable instructions.

The end unit 100 further includes a communications module 108, a GPSmodule 110, a power supply 112, an imaging device 114, and an interface120 for connecting one or more peripheral devices to the end unit 100.All of the components of the end unit 100 are connected or linked toeach other (electronically and/or data), either directly or indirectly,and are preferably retained within a single housing or casing with theexception of the imaging device 114 which may protrude from the housingor casing to allow for panning and tilting action, as will be discussedin further detail below. The communications module 108 is linked to thenetwork 150, and in certain embodiments may be implemented as a SIM cardor micro SIM, which provides data transfer functionality via cellularcommunication between the end unit 100 and the server 130 (and theprocessing subsystem 132) over the network 150.

The power supply 112 provides power to the major components of the endunit 100, including the processing unit 102, the communications module108, and the imaging device 114, as well as any additional components(e.g., sensors and illumination components) and peripheral devicesconnected to the end unit 100 via the interface 120. In a non-limitingimplementation, the power supply 112 is implemented as a battery, forexample a rechargeable battery, deployed to retain and supply charge asdirect current (DC) voltage. In certain non-limiting implementations,the output DC voltage supplied by the power supply 112 is approximately5 volts DC, but may vary depending on the power requirements of themajor components of the end unit 100.

In an alternative non-limiting implementation, the power supply 112 isimplemented as a voltage converter that receives alternating current(AC) voltage from a mains voltage power supply, and converts thereceived AC voltage to DC voltage, for distribution to the othercomponents of the end unit 100. An example of such a voltage converteris an AC to DC converter, which receives voltage from the mains voltagepower supply via a cable and AC plug arrangement connected to the powersupply 112. Note that the AC voltage range supplied by the mains voltagepower supply may vary by region. For example, a mains voltage powersupply in the United States typically supplies power in the range of100-120 volts AC, while a mains voltage power supply in Europe typicallysupplies power in the range of 220-240 volts AC.

In operation, the processing subsystem 132 commands the imaging device114 to capture images of the scene, and also commands the processingunit 102 to perform tasks. The control unit 139 may be implemented usinga processor, such as, for example, a microcontroller. Alternatively, theprocessor 136 of the image processing engine 134 may be implemented toexecute control functionality in addition to image processingfunctionality.

The end unit 100 may also include an illuminator 124 which providescapability to operate the end unit 100 in lighting environments, suchas, for example, nigh time or evening settings in which the amount ofnatural light is reduced, thereby decreasing visibility of the target34. The illuminator 124 may be implemented as a visible light source oras an infrared (IR) light source. In certain embodiments, theilluminator 124 is external from the housing of the end unit 100, andmay be positioned to the rear of the target 34 in order to illuminatethe target 34 from behind.

The imaging device 114 includes an image sensor 115 (i.e., detector) andan optical arrangement having at least one lens 116 which defines afield of view 118 of a scene to be imaged by the imaging device 114. Thescene to be imaged includes the target 34, such that the imaging device114 is operative to capture images of target 34 and projectile strikeson the target 34. The projectile strikes are detected by joint operationof the imaging device 114 and the processing subsystem 132, allowing thesystem 10 to detect strikes (i.e., projectile markings on the target 34)having a diameter in the range of 3-13 millimeters (mm).

The imaging device 114 may be implemented as a CMOS camera, and ispreferably implemented as a camera having pan-tilt-zoom (PTZ)capabilities, allowing for adjustment of the azimuth and elevationangles of the imaging device 114, as well as the focal length of thelens 116. In certain non-limiting implementations, the maximum pan angleis at least 90° in each direction, providing azimuth coverage of atleast 180°, and the maximum tilt angle is preferably at least 60°,providing elevation coverage of at least 120°. The lens 116 may includean assembly of multiple lens elements preferably having variable focallength so as to provide zoom-in and zoom-out functionality. Preferablythe lens 116 provides zoom of at least 2×, and in certain non-limitingimplementations provides zoom greater than 5×. As should be understood,the above range of angles and zoom capabilities are exemplary, andlarger or smaller angular coverage ranges and zoom ranges are possible.

The control subsystem 140 is configured to actuate the processingsubsystem 132 to commands the imaging device 114 to capture images, andto perform pan, tilt and/or zoom actions. The actuation commands issuedby the control subsystem 140 are relayed to the processing unit 102, viathe processing subsystem 132 over the network 150.

The system 10 is configured to selectively operate in two modalities ofoperation, namely a first modality and a second modality. The controlsubsystem 140 provides a control input, based on a user input command,to the end unit 100 and the processing subsystem 132 to operate thesystem 10 is the selected modality. In the first modality, referred tointerchangeably as a first mode, calibration modality or calibrationmode, the end unit 100 is calibrated in order to properly identifyprojectile strikes on the target 34. The calibration is based on therelative positioning between the end unit 100 and the target array 30.The firearm 20 should not be operated by a user of the system 10 duringoperation of the system 10 in calibration mode.

In the second modality, referred to interchangeably as a second mode,operational modality or operational mode, the processing subsystem 132identifies projectile strikes on the target 34, based on the imageprocessing techniques applied to the images captured by end unit 100,and provides statistical strike/miss data to the control subsystem 140.As should be understood, the firearm 20 is operated by the user of thesystem 10, in attempts to strike the target 34 one or more times. Whenthe user is ready to conduct target practice during a shooting sessionusing the system 10, the user actuates the system 10 to operate in theoperational mode via a control input command to the control subsystem140.

In certain embodiments, the calibration of the system 10 is performed byutilizing a bar code deployed on or near the target 34. As illustratedin FIGS. 2 and 4, the target 34 is positioned on a target holder 32,having sides 33 a-d. The target holder 32 may be implemented as astanding rack onto which the target 34 is be mounted. A bar code 36 ispositioned on the target holder 32, near the target 34, preferably onthe target plane and below the target 34 toward the bottom of the targetholder 32. In certain embodiments, the bar code 36 is implemented as atwo-dimensional bard code, more preferably a quick response code (QRC),which retains encoded information pertaining to the target 34 and thebar code 36. The encoded information pertaining to the bar code 36includes the spatial positioning of the bar code 36, the size (i.e., thelength and width) of the bar code 36, an identifier associated with thebar code 36, the horizontal (i.e., left and right) distance (x) betweenthe edges of the bar code 36 and the furthest horizontal points on theperiphery of the target 34 (e.g., the outer ring 35 a in the example inFIG. 2), and the vertical distance (y) between the bar code 36 and thefurthest vertical point on the periphery of the target 34. The encodedinformation pertaining to the target 34 includes size information of thetarget 34, which in the example of the target 34 in FIG. 2 may includethe diameter of each of the rings of the target 34, the distance fromthe center of the target 34 to the sides of the target holder 32, andspatial positioning information of the target 34 relative to the barcode 36. As shown in the FIG. 4, the bar code 36 is preferably centeredalong the vertical axis of the target 34 with respect to the center ring35 g, thereby resulting in the left and right distances between the barcode 36 and the furthest points on the outer ring 35 a being equal.

The encoded information pertaining to the target 34 and the bar code 36,specifically the horizontal distance x and the vertical distance y,serves as a basis for defining a coverage zone 38 of the target 34. Thehorizontal distance x may be up to approximately 3 meters (m), and thevertical distance y may be up to approximately 2.25 m. The coverage zone38 defines the area or region of space for which the processingcomponents of the system 10 (e.g., the processing subsystem 132) canidentify projectile strikes on the target 34. In the example illustratedin FIG. 4, the coverage zone 38 of the target 34 is defined as a regionhaving an area of approximately 2xy, and is demarcated by dashed lines.

Since the information encoded in the bar code 36 includes spatialpositioning information of the bar code 36 and the target 34 (relativeto the bar code 36), the spatial positioning of the bar code 36 and thetarget 34, in different reference frames, can be determined by either ofthe processing subsystem 132 or the processing unit 102. As such, theprocessor 104 preferably includes image processing capabilities, similarto the processor 136. Coordinate transformations may be used in order todetermine the spatial positioning of the bar code 36 and the target 34in the different reference frames.

Prior to operation of the system 10 in calibration or operational mode,the end unit 100 is first deployed proximate to the target array 30,such that the target 34 (or targets, as will be discussed in detail insubsequent sections of the document with respect to other embodiments ofthe present disclosure) is within the field of view 118 of the lens 116of the imaging device 114. For effective performance of the system 10 indetermining the projectile strikes on the target 34, the end unit 100 ispreferably positioned relative to the target array 30 such that the lineof sight distance between the imaging device 114 and the target 34 is inthe range of 1-5 m, and preferably such that the line of sight distancebetween the imaging device 114 and the bar code 36 is in the range of1.5-4 m. In practice, precautionary measures are taken in order to avoiddamage to the end unit 100 by inadvertent projectile strikes. In oneexample, the end unit 100 may be positioned in a trench or ditch, suchthat the target holder 32 is in an elevated position relative to the endunit 100. In such an example, the end unit 100 may be positioned up to50 centimeters (cm) below the target holder 32. In an alternativeexample, the end unit 100 may be covered or encased by a protectiveshell (not shown) constructed from a material having highstrength-to-weight ratio, such as, for example, Kevlar®. The protectiveshell is preferably open or partially open on the side facing thetarget, to allow unobstructed imaging of objects in the field of view118. In embodiments in which the end unit 100 operates with a singletarget 34, the end unit 100 may be mechanically attached to the targetholder 32.

The following paragraphs describe the operation of the system 10 incalibration mode. The operation of the system 10 in calibration mode isdescribed with reference to embodiments of the system 10 in which thetarget 34 is implemented as a physical target. However, as should beunderstood by one of ordinary skill in the art, operation of the system10 in calibration mode for embodiments of the system in which the target34 is implemented as a virtual target projected onto a screen orbackground by an image projector connected to the end unit 100 should beunderstood by analogy thereto.

In calibration mode, the end unit 100 is actuated by the controlsubsystem 140 to scan for bar codes that are in the field of view 118.In response to the scanning action, the end unit 100 recognizes barcodes in the field of view 118. The recognition of bar codes may beperformed by capturing an image of the scene in the field of view 118,by the imaging device 114, and identifying bar codes in the capturedimage.

With continued reference to FIG. 4, if the bar code 36 is in the fieldof view 118, the end unit 100 recognizes the bar code 36 in response tothe scanning action, and the encoded information stored in the bar code36, including the defined coverage zone 38 of the target 34, isextracted by decoding the bar code 36. In the case of bar coderecognition via image capture, the decoding of the bar code 36 may beperformed by analysis of the captured image by the processing unit 102,analysis of the captured image by the processing subsystem 132, or by acombination of the processing unit 102 and the processing subsystem 132.Such analysis may include analysis of the pixels of the captured barcode image, and decoding the captured image according to common QRCstandards, such as, for example, ISO/IEC 18004:2015.

As mentioned above, the field of view 118 is defined by the lens 116 ofthe imaging device 114. The imaging device 114 also includes a pointingdirection, based on the azimuth and elevation angles, which can beadjusted by modifying the pan and tilt angles of the imaging device 114.The pointing direction of the imaging device 114 can be adjusted toposition different regions or areas of a scene within the field of view118. If the spatial position of the target 34 in the horizontal andvertical directions relative to the field of view 118 does not match thedefined coverage zone 38, one or more imaging parameters of the imagingdevice 114 are adjusted until the bar code 36, and therefore the target34, is spatially positioned properly within the coverage zone 38. Inother words, if the defined coverage zone 38 of the target 34 is notinitially within the field of view 118, panning and/or tilting actionsare performed by the imaging device 114 based on calculated differencesbetween the pointing angle of the imaging device 114 and the spatialpositioning of the bar code 36.

FIG. 5A illustrates the field of view 118 of the imaging device 114 whenthe imaging device 114 is initially positioned relative to the targetholder 32. Based on the defined coverage zone 38, several imagingparameters, for example, the pan and tilt angles of the imaging device114, are adjusted to align the field of view 118 with the definedcoverage zone 38, as illustrated in FIG. 5B. The panning action of theimaging device 114 corresponds to horizontal movement relative to thetarget 34, while the tilting action of the imaging device 114corresponds to vertical movement relative to the target 34. As should beunderstood, the panning and tilting actions are performed while keepingthe base of the imaging device 114 at a fixed point in space.

In addition to aligning the field of view 118 with the coverage zone 38,the processing functionality of the system 10 (e.g., the processing unit102 and/or the processing subsystem 132) can determine the distance tothe target 34 from the end unit 100. As mentioned above, the encodedinformation pertaining to the bar code 36 includes the physical size ofthe bar code 36, which may be measured as a length and width (i.e., inthe horizontal and vertical directions). The number of pixels dedicatedto the portion of the captured image that includes the bar code 36 canbe used as an indication of the distance between the end unit 100 andthe bar code 36. For example, if the end unit 100 is positionedrelatively close to the bar code 36, a relatively large number of pixelswill be dedicated to the bar code portion 36 of the captured image.Similarly, if the end unit 100 is positioned relatively far from the barcode 36, a relatively small number of pixels will be dedicated to thebar code portion 36 of the captured image. As a result, a mappingbetween the pixel density of portions of the captured image and thedistance to the object being imaged can be generated by the processingunit 102 and/or the processing subsystem 132, based on the bar code 36size.

Based on the determined range from the end unit 100 to the bar code 36,the imaging device 114 may be actuated to adjust the zoom, to narrow orwiden the size of the imaged scene, thereby excluding objects outside ofthe coverage zone 38 from being imaged, or including regions at theperipheral edges of the coverage zone 38 in the imaged scene. Theimaging device 114 may also adjust the focus of the lens 116, to sharpenthe captured images of the scene.

Note that the zoom adjustment, based on the above-mentioned determineddistance, may successfully align the coverage zone 38 with desiredregions of the scene to be imaged if the determined distance is within apreferred range, which as mentioned above is preferably 1.5-4 m. If thedistance between the end unit 100 and the bar code 36 is determined tobe outside of the preferred range, the system 10 may not successfullycomplete calibration, and in certain embodiments, a message is generatedby the processing unit 102 or the processing subsystem 132, andtransmitted to the control subsystem 140 via the network 150, indicatingthat calibration failed due to improper positioning of the end unit 100relative to the target 34 (e.g., positioning too close to, or too farfrom, the target 34). The user of the system 10 may then physicallyreposition the end unit 100 relative to the target 34, and actuate thesystem 10 to operate in calibration mode.

According to certain embodiments, once the imaging parameters of theimaging device 114 are adjusted, in response to the recognition of thebar code 36, the imaging device 114 is actuated to capture an image ofthe coverage zone 38, and the captured image is stored in a memory, forexample, in the storage medium 106 and/or the server 130. The storedcaptured image serves as a baseline image of the coverage zone 38, to beused to initially evaluate strikes on the target 34 during operationalmode of the system 10. A message is then generated by the processingunit 102 or the processing subsystem 132, and transmitted to the controlsubsystem 140 via the network 150, indicating that calibration has beensuccessful, and that the system 10 is ready to operate in operationalmode.

By operating the system 10 in calibration mode, the imaging device 114captures information descriptive of the field of view 118. Thedescriptive information includes all of the image information as well asall of the encoded information extracted from the bar code 36 andextrapolated from the encoded information, such as the defined coveragezone 38 of the target 34. The descriptive information is provided to theprocessing subsystem 132 in response to actuation commands received fromthe control subsystem 140. Note that in the embodiments described above,the functions executed by the system 10 when operating in calibrationmode, in response to actuation by the control subsystem 140, areperformed automatically by the system 10. As will be discussed insubsequent sections of the present disclosure, in other embodiments ofthe system 10, operation of the system 10 in calibration mode may alsobe performed manually by a user of the system 10, via specific actuationcommands input to the control subsystem 140.

The following paragraphs describe the operation of the system 10 inoperational mode. The operation of the system 10 in operational mode isdescribed with reference to embodiments of the system 10 in which thetarget 34 is implemented as a physical target and the firearm 20 isimplemented as a live ammunition firearm that shoots live ammunition.However, as should be understood by one of ordinary skill in the art,operation of the system 10 in operational mode for embodiments of thesystem in which the target 34 is implemented as a virtual targetprojected onto a screen or background by an image projector connected tothe end unit 100 should be understood by analogy thereto.

In operational mode, the end unit 100 is actuated by the controlsubsystem 140 to capture a series of images of the coverage zone 38 at apredefined image capture rate (i.e., frame rate). Typically, the imagecapture rate is 25 frames per second (fps), but can be adjusted tohigher or lower rates via user input commands to the control subsystem140. Individual images in the series of images are compared with one ormore other images in the series of images to identify changes betweenimages, in order to determine strikes on the target 34 by the projectile22. According to certain embodiments, the image comparison is performedby the processing subsystem 132, which requires the end unit 100 totransmit each captured image to the server 130, over the network 150,via the communications module 108. Each image may be compressed prior totransmission to reduce the required transmission bandwidth. As such, theimage comparison processing performed by the processing subsystem 132may include decompression of the images. In alternative embodiments, theimage comparison may be performed by the processing unit 102. However,it may be advantageous to offload as much of the image processingfunctionality as possible to the processing subsystem 132 in order toreduce the complexity of the processing unit 102, thereby lessening thesize, weight and power (SWAP) requirements of the end unit 100.

It is noted that the terms “series of images” and “sequence of images”may be used interchangeably throughout this document, and that theseterms carry with them an inherent temporal significance such thattemporal order is preserved. In other words, a first image in the seriesor sequence of images that appears prior to a second image in the seriesor sequence of images, implies that the first image was captured at atemporal instance prior to the second image.

Refer now to FIGS. 6A-6E, an example of five images 60 a-e of thecoverage zone 38 captured by the imaging device 114. The images capturedby the imaging device 114 are used by the processing subsystem 132, inparticular the image processing engine 134, in a process to detect oneor more strikes on the target 34 by projectiles fired by the firearm 20.Generally speaking, the process relies on comparing a current imagecaptured by the imaging device 114 with one or more previous imagescaptured by the imaging device 114.

The first image 60 a (FIG. 6A) is the baseline image of the coveragezone 38 captured by the imaging device 114 during the operation of thesystem 10 in calibration mode. In the example illustrated in FIG. 6A,the baseline image depicts the target 34 without any markings fromprevious projectile strikes (i.e., a clean target). However, the targetmay have one or more markings from previous projectile strikes.

The second image 60 b (FIG. 6B) represents one of the images in theseries of images captured by the imaging device 114 during operation ofthe system 10 in operational mode. As should be understood, each of theimages in the series of images captured by the imaging device 114 duringoperation of the system 10 in operational mode are captured at temporalinstances after the first image 60 a. The first and second images 60 a-bare transmitted to the processing subsystem 132 by the end unit 100,where the image processing engine 134 analyzes the two images todetermine if a change occurred in the scene captured by the two images.In the example illustrated in FIG. 6B, the second image 60 b isidentical to the first image 60 a, which implies that although the userof the system 10 may have begun operation of the firearm 20 (i.e.,discharging of the projectile 22), the user has failed to strike thetarget 34 during the period of time after the first image 60 a wascaptured. The image processing engine 134 determines that no change tothe scene occurred, and therefore a strike on the target 34 by theprojectile 22 is not detected. Accordingly, the second image 60 b isupdated as the baseline image of the coverage zone 38.

The third image 60 c (FIG. 6C) represents a subsequent image in theseries of images captured by the imaging device 114 during operation ofthe system 10 in operational mode. The third image 60 c is captured at atemporal instance after the images 60 a-b. The image processing engine134 analyzes the second and third images 60 b-c to determine if a changeoccurred in the scene captured by the two images. As illustrated in FIG.6C, firing of the projectile 22 results in a strike on the target 34,illustrated in FIG. 6C as a marking 40 on the target 34. The imageprocessing engine 134 determines that a change to the scene occurred,and therefore a strike on the target 34 by the projectile 22 isdetected. Accordingly, the second image 60 b is updated as the baselineimage of the coverage zone 38.

The fourth image 60 d (FIG. 6D) represents a subsequent image in theseries of images captured by the imaging device 114 during operation ofthe system 10 in operational mode. The fourth image 60 d is captured ata temporal instance after the images 60 a-c. The image processing engine134 analyzes the third and fourth images 60 c-d to determine if a changeoccurred in the scene captured by the two images. As illustrated in FIG.6D, the fourth image 60 d is identical to the third image 60 c, whichimplies that the user has failed to strike the target 34 during theperiod of time after the third image 60 d was captured. The imageprocessing engine 134 determines that no change to the scene occurred,and therefore a strike on the target 34 by the projectile 22 is notdetected. Accordingly, the fourth image 60 d is updated as the baselineimage of the coverage zone 38.

The fifth image 60 e (FIG. 6E) represents a subsequent image in theseries of images captured by the imaging device 114 during operation ofthe system 10 in operational mode. The fifth image 60 e is captured at atemporal instance after the images 60 a-d. The image processing engine134 analyzes the fourth and fifth images 60 d-e to determine if a changeoccurred in the scene captured by the two images. As illustrated in FIG.6E, firing of the projectile 22 results in a second strike on the target34, illustrated in FIG. 6E as a second marking 42 on the target 34. Theimage processing engine 134 determines that a change to the sceneoccurred, and therefore a strike on the target 34 by the projectile 22is detected. Accordingly, the second image 60 b is updated as thebaseline image of the coverage zone 38.

As should be apparent, the process for detecting strikes on the target34 may continue with the capture of additional images and the comparisonof such images with previously captured images.

The term “identical” as used above with respect to FIGS. 6A-6E refers toimages which are determined to be closely matched by the imageprocessing engine 134, such that a change to the scene is not detectedby the image processing engine 134. The term “identical” is not intendedto limit the functionality of the image processing engine 134 todetecting changes to the scene only if the corresponding pixels betweentwo images have the same value.

With respect to the above described process for detecting strikes on thetarget 34, the image processing engine 134 is preferably configured toexecute one or more image comparison algorithms, which utilize one ormore computer vision and/or image processing techniques. In one example,the image processing engine 134 may be configured to execute keypointmatching computer vision algorithms, which rely on picking points,referred to as “key points”, in the image which contain more informationthan other points in the image. An example of keypoint matching is thescale-invariant feature transform (SIFT), which can detect and describelocal features in images, described in U.S. Pat. No. 6,711,293.

In another example, the image processing engine 134 may be configured toexecute histogram image processing algorithms, which bin the colors andtextures of each captured image into histograms and compare thehistograms to determine a level of matching between compared images. Athreshold may be applied to the level of matching, such that levels ofmatching above a certain threshold provide an indication that thecompared images are nearly identical, and that levels of matching belowthe threshold provide an indication that the compared images aredemonstrably different.

In yet another example, the image processing engine 134 may beconfigured to execute keypoint decision tree computer vision algorithms,which relies on extracting points in the image which contain moreinformation, similar to SIFT, and using a collection decision tree toclassify the image. An example of keypoint decision tree computer visionalgorithms is the features-from-accelerated-segment-test (FAST), theperformance of which can be improved with machine learning, as describedin “Machine Learning for High-Speed Corner Detection” by E. Rosten andT. Drummond, Cambridge University, 2006.

As should be understood, results of such image comparison techniques maynot be perfectly accurate, resulting in false detections and/or misseddetections, due to artifacts such as noise in the captured images, anddue to computational complexity. However, the selected image comparisontechnique may be configured to operate within a certain tolerance valueto reduce the number of false detections and missed detections.

Note that the image capture rate, nominally 25 fps, is typically fasterthan the maximum rate of fire of the firearm 20 when implemented as anon-automatic weapon. As such, the imaging device 114 most typicallycaptures images more frequently than shots fired by the firearm 20.Accordingly, when the system 10 operates in operational mode, theimaging device 114 will typically capture several identical images ofthe coverage zone 38 which correspond to the same strike on the target34. This phenomenon is exemplified in FIGS. 6B-6E, where no change inthe scene is detected between the third and fourth images 60 c-d.

Although embodiments of the system 10 as described thus far havepertained to an image processing engine 134 that compares a currentimage with a previous image to identify changes in the scene, therebydetecting strikes on the target 34, other embodiments are possible inwhich the image processing engine 134 is configured to compare thecurrent image with more than one previous image, to reduce theprobability of false detection and missed detection. Preferably, thepreviously captured images used for the comparison are consecutivelycaptured images. For example, in a series of N images, if the currentimage is the k^(th) image, the in previous images are the k−1, k−2, . .. , k−m images. In such embodiments, no decision on strike detection ismade for the first m images in the series of images.

Each comparison of the current image to a group of previous images maybe constructed from subsets of in pairwise comparisons, the output ofeach pairwise comparison being input to a majority logic decision.Alternatively, the image processing engine 134 may average the pixelvalues of the in previous images to generate an average image, which canbe used to compare with the current image. The averaging may beimplemented using standard arithmetic averaging or using weightedaveraging.

During operational mode, the system 10 collects and aggregates strikeand miss statistical data based on the strike detection performed by theprocessing subsystem 132. The strike statistical data includes accuracydata, which includes statistical data indicative of the proximity of thedetected strikes to the rings 35 a-g of the target 34. The evaluation ofthe proximity to the rings 35 a-g of the target 34 is based on thecoverage zone 38 and the spatial positioning information obtained duringoperation of the system 10 in calibration mode.

The statistical data collected by the processing subsystem 132 is madeavailable to the control subsystem 140, via, for example, push request,in which the user of the system 10 actuates the control subsystem 140 tosend a request to the server 130 to transmit the statistical results oftarget training activity to the control subsystem 140 over the network150. The statistical results may be stored in a database (not shown)linked to the server 130, and may be stored for each target trainingsession of the user of the end unit 100. As such, the user of the endunit 100 may request to receive statistical data from a current targettraining session and a previous target training session to gaugeperformance improvement. Such performance improvement may also be partof the aggregated data collected by the processing subsystem 132. Forexample, the processing subsystem 132 may compile a statistical historyof a user of the end unit 100, summarizing the change in target accuracyover a period of time.

Although the embodiments of the system 10 as described thus far havepertained to an end unit 100, a processing subsystem 132 and a controlsubsystem 140 operating jointly to identify target strikes from afirearm implemented as a live ammunition firearm that shoots liveammunition, other embodiments are possible, as mentioned above, in whichthe firearm is implemented as a light pulse based firearm which producesone or more pulses of coherent light (e.g., laser light).

Refer now to FIG. 8, a firearm 20′ implemented as a light pulse basedfirearm. The firearm 20′ includes a light source 21 for producing one ormore pulses of coherent light (e.g., laser light), which are output inthe form of a beam 23. In such embodiments, the beam 23 acts as theprojectile of the firearm 20′. According to certain embodiments, thelight source 21 emits visible laser light at a pulse length ofapproximately 15 milliseconds (ms) and at a wavelength in the range of635-655 nanometers (nm).

In other embodiments, the light source 21 emits IR light at a wavelengthin the range of 780-810 nm. In such embodiments, in order to performdetection of strikes on the target by the beam 23, the end unit 100 isequipped with an IR image sensor 122 (referred to hereinafter as IRsensor 122) that is configured to detect and image the IR beam 23 thatstrikes the target 34. The processing components of the system 10 (i.e.,the processing unit 102 and the processing subsystem 132) identify theposition of the beam 23 strike on the target 34 based on the detectionby the IR sensor 122 and the correlated position of the beam 23 in theimages captured by the imaging device 114. The IR sensor may beimplemented as an IR camera that is separate from the imaging device114. Alternatively, the IR sensor 122 may be housed together with theimage sensor 115 as part of the imaging device 114. In such aconfiguration, the image sensor 115 and the IR sensor 122 preferablyshare resources, such as, for example, the lens 116, to ensure that thesensors 115, 122 are exposed to the same field of view 118.

The process to detect one or more strikes on the target 34 is differentin embodiments in which the firearm 20′ is implemented as a lightpulse-based firearm as compared to embodiments in which the firearm 20is implemented a live ammunition firearm that shoots live ammunition.For example, each current image is compared with the last image in whichno strike on the target 34 by the beam 23 was detected by the processingsubsystem 132. If a strike on the target 34 by the beam 23 is detectedby the processing subsystem 132, the processing subsystem 132 waitsuntil an image is captured in which the beam 23 is not present in theimage, essentially resetting the baseline image. This process avoidsdetecting the same laser pulse multiple times in consecutive frames,since the pulse length of the beam 23 is much faster than the imagecapture rate of the imaging device 114.

In order to execute the appropriate process to detect one or morestrikes on the target 34 when the system 10 operates in operationalmode, the bar code 36 preferably conveys to the system 10 the type offirearm 20, 20′ to be used in operational mode. As such, according tocertain embodiments, in addition to the bar code 36 retaining encodedinformation pertaining to the target 34 and the bar code 36, the barcode 36 also retains encoded information related to the type of firearmto be used in the training session. Accordingly, the user of the system10 may be provided with different bar codes, some of which are encodedwith information indicating that the training session uses a firearmthat shoots live ammunition, and some of which are encoded withinformation indicating that the training session uses a firearm thatemits laser pulses. The user may select which bar code is to be deployedon the target holder 32 prior to actuating the system 10 to operate incalibration mode. The bar code 36 deployed on the target holder 32 maybe interchanged with another bar code, thereby allowing the user of thesystem 10 to deploy a bar code encoded with information specifying thetype of firearm. In calibration mode, the type of firearm is extractedfrom the bar code, along with the above described positionalinformation.

Although the embodiments of the system 10 as described thus far havepertained to an end unit 100 operating in tandem with processingcomponents and a control system to identify target strikes, otherembodiments are possible in which the end unit 100 additionally providescapabilities for interactive target training sessions. As mentionedabove, and as illustrated in FIG. 3, the end unit 100 includes aninterface 120 for connecting one or more peripheral devices to the endunit 100. The interface 120, although illustrated as a single interface,may represent one or more interfaces, each configured to connect adifferent peripheral device to the end unit 100.

Refer now to FIG. 9, a simplified block diagram of the end unit 100connected with several peripheral devices, including an image projectionunit 160 and an audio unit 162. The image projection unit 160 may beimplemented as a standard image projection system which can project animage or a sequence of images against a background, for example aprojection screen constructed of thermoelastic material. The imageprojection unit 160 can be used in embodiments in which the target 34 isbe implemented as a virtual target. According to certain embodiments,the image projection unit 160 projects an image of the bar code 36 aswell as an image of the target 34. In such embodiments, the system 10operates in calibration and operational modes, similar to as describedabove.

The audio unit 162 may be implemented as a speaker system configured toplay audio from an audio source embedded in the end unit 100. Theprocessor 104, for example, may be configured to provide audio to theaudio unit 162. The audio unit 162 and the image projection unit 160 areoften used in tandem to provide an interactive training scenario whichsimulates real-life combat or combat-type situations. In suchembodiments the bar code 36 also retains encoded information pertainingto the type of target 34 and the type of training session. As an exampleof such a training scenario, the image projection unit 160 may project avideo image of an armed hostage taker holding a hostage. The audio unit162 may provide audio synchronized with the video image projected by theimage projection unit 160. In such a scenario, the hostage taker istreated by the system 10 as the target 34. As such, the region of thecoverage zone 38 occupied by the target 34 changes dynamically as thevideo image of the hostage taker moves as the scenario progresses, andis used by the processing subsystem 132 to evaluate projectile strikes.

In response to a detected projectile strike or miss on the definedtarget (e.g., the hostage taker or other target object projected by theimage projection unit 160), the system 10 may actuate the imageprojection unit 160 to change the projected image. For example, if theimage projection unit 160 projects an image of a hostage taker holding ahostage, and the user fired projectile fails to strike the hostagetaker, the image projection unit 160 may change the projected image todisplay the hostage taker attacking the hostage.

As should be apparent, the above description of the hostage scenario isexemplary only, and is intended to help illustrate the functionality ofthe system 10 when using the image projection unit 160 and otherperipheral devices in training scenarios.

With continued reference to FIG. 9, the end unit 100 may also beconnected to a motion control unit 164 for controlling the movement ofthe target 34. According to certain embodiments, the motion control unit164 is physically attached to the target 34 thereby providing amechanical coupling between the end unit 100 and the target 34. Themotion control unit 164 may be implemented as a mechanical drivingarrangement of motors and gyroscopes, allowing multi-axis translationaland rotational movement of the target 34. The motion control unit 164receives control signals from the control unit 139 via the processingunit 102 to activate the target 34 to perform physical actions, e.g.,movement. The control unit 139 provides such control signals to themotion control unit 164 in response to events, for example, targetstrikes detected by the image processing engine 134, or direct inputcommands by the user of the system 10 to move the target 34.

Although the embodiments of the system 10 as described thus far havepertained to operation with a target array 30 that includes a singletarget, other embodiments are possible in which the target array 30includes multiple targets. Refer now to FIG. 10, an exemplaryillustration of a target array 30 that includes three targets, namely afirst target 34 a, a second target 34 b, and a third target 34 c. Eachtarget is mounted to a respective target holder 32 a-c, that has arespective bar code 36 a-c positioned near the respective target 34 a-c.The boundary area of the target array 30 is demarcated with a dottedline for clarity.

The use of multiple targets allows the user of the system 10 toselectively choose and alternate which of the individual targets to usefor training. Although the targets 34 a-c as illustrated in FIG. 10appear identical and evenly spaced relative to each other, each targetmay be positioned at a different distance from the end unit 100, and ata different height relative to the end unit 100.

Note that the illustration of three targets in the target array 30 ofFIG. 10 is for example purposes only, and should not be taken to limitthe number of targets in the target array 30 to a particular value. Inpractice, a single target array 30 may include up to ten such targets.

Similar to as discussed above, prior to operation of the system 10 incalibration or operational mode, the end unit 100 is first deployedproximate to the target array 30, such that the targets 34 a-c arewithin the field of view 118 of the lens 116 of the imaging device 114.As discussed above, in calibration mode, the end unit 100 is actuated bythe control subsystem 140 to scan for bar codes that are in the field ofview 118. In response to the scanning action, the end unit 100recognizes the bar codes 36 a-c in the field of view 118, via forexample image capture by the imaging device 114 and processing by theprocessing unit 102 or the processing subsystem 132. In response to therecognition of the bar codes 36 a-c, the control subsystem 140 receivesfrom the end unit 100 an indication of the number of targets in thetarget array 30. For example, in the three-target deployment illustratedin FIG. 10, the control subsystem 140 receives an indication that thetarget array 30 includes three targets in response to the recognition ofthe bar codes 36 a-c. Furthermore, each of the bar codes 36 a-c isuniquely encoded to include an identifier associated with the respectivebar codes 36 a-c. This allows the control subsystem 140 to selectivelychoose which of the targets 36 a-c to use when the system 10 operates inoperational mode.

The operation of the system 10 in calibration mode in situations inwhich the target array 30 includes multiple targets, for example asillustrated in FIG. 10, is generally similar to the operation of thesystem 10 in calibration mode in situations in which the target array 30includes a single target, for example as illustrated in FIGS. 2 and4-5B. As discussed above, according to certain embodiments, theinformation descriptive of the field of view 118 that is captured by theimaging device 114 is provided to the processing subsystem 132 inresponse to actuation commands received from the control subsystem 140.The descriptive information includes all of the image information aswell as all of the encoded information extracted from the bar codes 36a-c and extrapolated from the encoded information, which includes thedefined coverage zone for each of the targets 34 a-c. As noted above,the encoded information includes an identifier associated with each ofthe respective bar codes 36 a-c, such that each of targets 34 a-c isindividually identifiable by the system 10. According to certainembodiments, the coverage zone for each of the targets 34 a-c may bemerged to form a single overall coverage zone. In such embodiments, astrike on any of the targets is detected by the system 10, along withidentification of the individual target that was struck.

According to certain embodiments, when operating the system 10 inoperational mode, the user of the system 10 is prompted, by the controlsubsystem 140, to select one of the targets 34 a-c for which the targetraining session will take place. The control subsystem 140 actuates theend unit 100 to capture a series of images, and the processing subsystem132 analyzes regions of the images corresponding to coverage zone of theselected target. The analyzing performed by the processing subsystem 132includes the image comparison, performed by the image processing engine134, as described above.

Although the embodiments of the system 10 as described thus far havepertained to a control subsystem and a processing subsystem linked, viaa network, to a single end unit (i.e., the end unit 100), otherembodiments are possible in which the control subsystem 140 and theprocessing subsystem 132 are linked to multiple end units 100 a-N, asillustrated in FIG. 11, with the structure and operation of each of theend units 100 a-N being similar to that of the end unit 100. In thisway, a single control subsystem can command and control an array of endunits deployed in different geographic location.

The embodiments of the control subsystem 140 of the system 10 of thepresent disclosure have been described thus far in terms of the logicalcommand and data flow between the control subsystem 140 and the end unit100 and the processing subsystem 132. The control subsystem 140 may beadvantageously implemented in ways which allow for mobility of thecontrol subsystem 140 and effective accessibility of the data providedto the control subsystem 140. As such, according to certain embodiments,the control subsystem 140 is implemented as a management application 242executable on a mobile communication device. The management application242 may be implemented as a plurality of software instructions orcomputer readable program code executed on one or more processors of themobile communication device. Examples of mobile communication devicesinclude, but are not limited to, smartphones, tablets, laptop computers,and the like. Such devices typically included hardware and softwarewhich provide access to the network 150, which allow transfer of data toand from the network 150.

Refer now to FIG. 12, a non-limiting illustration of the managementapplication 242 executable on a mobile communication device 240. Themanagement application 242 provides a command and control interfacebetween the user and the components of the system 10. The managementapplication 242, as illustrated in FIG. 12, includes a display area 244with a home screen having multiple icons 248 for commanding the system10 to take actions based on user touchscreen input. The display area 244also includes a display region 246 for displaying information inresponse to commands input to the system 10 by the user via themanagement application 242. The management application 242 is preferablydownloadable via an application server and executed by the operatingsystem of the mobile communication device 240.

One of the icons 248 provides an option to pair the managementapplication 242 with an end unit 100. The end unit 100 to be paired maybe selectable based on location, and may require an authorization codeto enable the pairing. The location of the end unit 100 is provided tothe server 130 and the control subsystem 140 (i.e., the managementapplication 242) via the GPS module 110. The pairing of the managementapplication 242 and the end unit 100 is performed prior to operating theend unit in calibration or operational modes. As noted above, multipleend units may be paired with the control subsystem 140, and thereforewith the management application 242. A map displaying the locations ofthe paired end units may be displayed in the display region 246. Thelocations may be provided by the GPS module 110 of each end unit 100, inresponse to a location request issued by the management application 242.

Upon initial download of the management application 242, no end unitsare typically paired with the management application 242. Therefore, oneor more of the remaining icons 248 may be used to provide the user ofthe system 10 with information about the system 10 and system settings.For example, a video may be displayed in the display region 246providing user instructions on how to pair the management application242 with end units, how to operate the system 10 in calibration andoperational modes, how to view statistical strike/miss data, how togenerate and download interactive training scenarios, and other tasks.

Preferably, a subset of the icons 248 include numerical identifierscorresponding to individual end units to which the managementapplication 242 is paired. Each of the icons 248 corresponding to anindividual end unit 100 includes status information of the end unit 100.The status information may include, for example, power status andcalibration status.

As mentioned above, the end unit 100 includes a power supply 112, whichin certain non-limiting implementations may be implemented as a batterythat retains and supplies charge. The icon 248 corresponding to the endunit 100 displays the charge level, for example, symbolically ornumerically, of the power supply 112 of the end unit 100, whenimplemented as a battery.

The calibration status of the end unit 100 may be displayed symbolicallyor alphabetically, in order to convey to the user of the system 10whether the end unit 100 requires operation in calibration mode. If thecalibration status of the end unit 100 indicates that the end unit 100requires calibration, the user may input a command to the managementapplication 242, via touch selection, to calibrate the end unit 100. Inresponse to the user input command, the system 10 operates incalibration mode, according to the processes described in detail above.Optionally, the user may manually calibrate the end unit 100 by manuallyentering the distance of the end unit 100 from the target 34, manuallyentering the dimensions of the desired coverage zone 38, and manuallyadjusting the imaging parameters of the imaging device 114 (e.g., zoom,focus, etc.). Such manual calibration steps may be initiated by the userinputting commands to the management application 242, via for exampletouch selection. Typically, the user of the system 10 is provided withboth calibration options, and selectively chooses the calibration optionbased on an input touch command. The manual calibration option may alsobe provided to the user of the system 10 if the end unit 100 fails toproperly read the bar code 36, due to system malfunction or otherreasons, or if the bar code 36 is not deployed on the target holder 32.Note that the manual calibration option may be used to advantage inembodiments of the system 10 in which the target 34 is be implemented asa virtual target projected onto a screen or background by the imageprojection unit 160, as described above with reference to FIG. 9.

As mentioned above, each end unit 100 that is paired with the managementapplication 242 has an icon 248, preferably a numerical icon, displayedin display area 244. According to certain embodiments, selection of anicon 248 that corresponds to an end unit 100 changes the display of themanagement application 242 from the home screen to an end unit detailsscreen associated with that end unit 100.

Referring to FIG. 13, a non-limiting illustration of the details screen.The details screen preferably includes additional icons 250corresponding to the targets of the target array 30 proximate to whichthe end unit 100 is deployed. As mentioned above, each of the targets 34of the target array 30 includes an assigned identifier encoded inrespective the bar code 36. The assigned identifier is preferably anumerical identifier, and as such, the icons corresponding to thetargets 34 are represented by the numbers assigned to the targets 34.Referring again to example illustrated in FIG. 10, the first target 34 amay be assigned the identifier ‘1’, the second target 34 b may beassigned the identifier ‘2’, and the third target 34 c may be assignedthe identifier ‘3’. Accordingly, the details screen displays three icons250 labeled as ‘1’, ‘2’, and ‘3’. The details screen may also display animage, as captured by the imaging device 114, of the target 34 in thedisplay region 246.

According to certain embodiments, selection of one of the icons 250displays target strike data and statistical data, that may be currentand/or historical data, indicative of the proximity of the detectedstrikes on the selected target 34. The data may be presented in variousformats, such as, for example, tabular formats, and may displayed in thedisplay region 246 or other regions of the display area 244. In anon-limiting implementation, the target strike data is presentedvisually as an image of the target 34 and all of the points on thetarget 34 for which the system 10 detected a strike from the projectile22. In this way, the user of system 10 is able to view a visual summaryof a target shooting session.

Note that the functionality of the management application 242 may alsobe provided to the user of the system 10 through a web site, which maybe hosted by a web server (not shown) linked to the server 130 over thenetwork 150.

As discussed throughout the present disclosure, the imaging device 114is operative to capture images of the scene, and more specificallyimages of the target 34, when the system 10 operates in both calibrationand operational modes. In the previously described embodiments, theimages captured by the imaging device 114 are visible light images. Onedrawback of capturing visible light images during operation inoperational mode is that detection of projectile strikes on the target34 by the relevant processing systems—based on the images of the target34 captured by the imaging device 114—may be limited due in part tolighting and shadow effects on the target. This may become particularlyproblematic when the target is a virtual target that is part of avirtual training scenario, for example a scenario projected onto aprojection screen by the image projection unit 160, where the processingsystem identifies projectile strikes by detecting holes in theprojection screen created by the projectiles, but where such holes maybe in dark or shaded regions of the projection screen which makes theholes difficult to discern from the dark or shaded regions.

One solution which overcomes such drawbacks is the use of processorsthat implement more advanced processing technologies that can moreeasily differentiate between holes and dark or shaded regions on theprojection screen. However, such solutions require more complexprocessing architectures, which can become prohibitively expensive.

Another possible solution, discussed in previously describedembodiments, is the deployment of a dedicated IR sensor 122 whichdetects and images IR light. IR imaging of a target makes the projectilestrikes on the target (such as holes in the projection screen) moreeasily distinguishable from dark or shaded regions of the target orprojection screen. However, utilizing an IR image sensor for imagecapture in calibration mode of the system 10 is not ideal as IR imagesmay not provide high enough image resolution in order to accuratelyextract target spatial information and coverage zone. Therefore, such adedicated IR image sensor should be used in combination with the imagesensor 115, where the image sensor 115 is used in calibration mode andthe IR image sensor is used in operational mode. However, this solutionrequires two separate image sensors which is increases cost.Furthermore, the use of one image sensor in calibration mode and anotherimage sensor in operational mode requires that the processing componentsof the system 10 that control the image sensors (e.g., the processingunit 102 and/or the processing subsystem 132) actively switch the imagesensors on and off during operation of the system 10, which increasesprocessing and control complexity. However, it is noted that the presentdisclosure does not preclude embodiments which utilize the image sensor115 and the IR sensor 122 in tandem.

In order to provide a cost-effective and low-complexity solution thatyields accurate projectile strike detection performance, the presentembodiments utilize the image sensor 115 of the imaging device 114 tocapture visible light images of the target 34 during calibration mode,and then utilize the same image sensor 115 of the same imaging device114 to capture infrared (IR) images of the target 34 during operationalmode. The key is to employ an IR positioning mechanism, operativelycoupled to the end unit 100, that can position an IR filter in and outof the optical path from the scene (i.e., the target 34) to the imagesensor 115 in accordance with the mode of the operation of the system10. It is noted that in such embodiments, the image sensor 115 issensitive to all radiation in wavelengths between approximately 350 nmand approximately 1000 nm, i.e., is sensitive to radiation in thevisible light regions (350 nm-700 nm) and IR regions (700 nm-1000 nm) ofthe electromagnetic spectrum. Parenthetically, most commercial off theshelf (COTS) cameras include image sensors that are sensitive toradiation in the IR and visible light regions of the electromagneticspectrum. However, such cameras typically contain dichroic filters, inthe form of hot mirrors, which block IR radiation from reaching theimage sensor by reflecting incoming IR light. Therefore, when utilizinga COTS camera as the imaging device 114, the IR blocking dichroic filtershould be removed or disabled in order to provide the imaging device 114with the capability of capturing full spectral images.

Refer now to FIG. 14, a schematic side view representation of the endunit 100 of the system 10 deployed against the target 34 according tothe present embodiments in which the system 10 further includes an IRfilter assembly 300 that is deployed to selectively position an IRfilter in and out of a portion of the optical path from the scene (to beimaged by the imaging device 114) to the image sensor 115. As willbecome apparent from the following description, one of the advantages ofthe present embodiments is that the IR filter assembly 300 can bedeployed as an add-on component to an existing imaging device wherebythe portion of the optical path is between the scene and the lens of theimaging device, and does not require a more complex imaging device inwhich a switchable IR filter is deployed internal to the imaging deviceso as to be positionable in and out of the portion of the optical paththat is between the imaging lens and the image sensor. However, it isnoted that the present disclosure does not preclude embodiments of suchaforementioned more complex solutions.

Within the context of the present disclosure, the term “IR filter”generally refers to a filter that passes IR light and blocks non-IRlight. In other words, IR filters, within the context of this document,pass radiation at wavelengths in the IR region of the electromagneticspectrum and block radiation at wavelengths outside of the IR region ofthe electromagnetic spectrum. In particularly preferred embodiments, theIR filter is configured to pass light in a particular sub-region of theIR region, namely the near-infrared (NIR) region, which nominallyincludes wavelengths in the range between approximately 750 nm and 1400nm, but for the purposes of the present invention preferably extendsdown to include wavelengths at the upper end of the visible light region(approximately 700 nm). Even more preferably, the IR filter isconfigured to pass light having wavelengths in the range betweenapproximately 700 nm and 1000 nm.

In certain cases, such as when the system 100 is deployed in outdoorenvironments, the IR filter most preferably has a particularly narrowspectral passband in the NIR region. By way of introduction, sunlight atwavelengths of approximately 942 nm is typically absorbed by theatmosphere, and therefore ambient sunlight illumination at 942 nm tendsto not impinge on optical sensors, or to impinge on optical sensors at arelatively low intensity compared to the intensity of light that is tobe imaged by the sensor. Therefore, performing IR imaging of objects inoutdoor environments at wavelengths in the vicinity of 942 nm tends toyield high-quality IR images. Hence, it is preferable to implement theIR filter 302 with a passband centered closely around approximately 942nm, in particular when the system 100 is deployed outdoors. In onenon-limiting example, the IR filter 302 is implemented with a passbandin the range between approximately 935 nm and 945 nm (i.e., the IRfilter only passes light having wavelengths in the range of 935-945 nm).

With continued reference to FIG. 14, refer now to FIGS. 15A-16B, variousschematic views of the IR filter assembly 300 deployed relative to theimaging device 114. The IR filter assembly 300 includes an IR filter 302and an IR filter positioning mechanism 310 (referred to hereinafter aspositioning mechanism 310) that is operative to selectivelymove/position the IR filter 302 in and out of a portion of the opticalpath (generally designated 350 in FIGS. 16A and 16B) from the scene tothe imaging device 114, and more particularly from the scene to theimage sensor 115. The optical path 350 is defined by the opticalarrangement (lens 116) of the imaging device 114. Generally speaking,the scene includes the target 34 when the end unit 100 is properlydeployed and positioned adjacent to the target 34 such that the target34 is in the field of view 118.

Parenthetically, although the IR filter 302 is generally configured, asmentioned above, to pass light (radiation) in the IR region of theelectromagnetic spectrum and block light outside of the IR region of theelectromagnetic spectrum (i.e., less than 700 nm and greater than 1000nm, reducing the spectral passband of the IR filter to pass a particularnarrow spectral region of the infrared range has been found toparticularly useful when deploying the system 100 in outdoorenvironments. In a particularly preferred but non-limitingimplementation, the IR filter 302 is configured to pass light havingwavelengths in a narrow range centered around 942 nm, for example935-945 nm. Sunlight at wavelengths of 942 nm is typically absorbed bythe atmosphere, and therefore ambient sunlight illumination tends to notimpinge on optical sensors, or to impinge on optical sensors and lowintensity compared to the intensity of light that is to be imaged by thesensor. Therefore, performing IR imaging of objects at wavelengths inthe vicinity of 942 nm yields high-quality IR images. Hence, it ispreferable to implement the IR filter 302 with a passband centeredclosely around approximately 942 nm. For example, the IR filter 302 canpreferably be implemented to block light having wavelengths outside ofthe 935-945 nm range.

The optical path 350 from the scene to the image sensor 114 is generallydefined herein as the region of space through which light from the scenecan traverse directly to and through the imaging device 114 so as to beimaged by the lens 116 onto the image sensor 115. The optical path 350overlaps entirely with the field of view 118 defined by the lens 116,and includes two optical portions. A first optical path portion(generally designated 352) between the scene and the lens 116, and asecond optical path portion (generally designated 354) between the lens116 and the image sensor 115. In the preferred but non-limitingembodiments illustrated in FIGS. 14-16B, the IR filter 302 ispositionable a short distance in front of the lens 116, and between thelens 116 and the scene, i.e., the portion of the optical path 350 is theoptical path portion 352 between the scene and the lens 116.

When the IR filter 302 is positioned in the optical path 350, all of thelight from the scene within the field of view 118 passes through the IRfilter 302, such that the visible light within the field of view 118 isblocked by the IR filter 302 and only the IR light within the field ofview 118 reaches the image sensor 115. Conversely, when the IR filter302 is positioned out of the optical path 350, none of the light fromthe scene passes through the IR filter 302 such that all of the light(both visible and IR) from the scene within the field of view 118reaches the image sensor 115.

The positioning mechanism 310 includes an electro-mechanical actuator312 in mechanical driving relationship with the IR filter 302. Manyactuator configurations are contemplated herein, including, but notlimited to, rotary actuators and linear actuators. In the non-limitingimplementation illustrated in the drawings, the actuator 312 isimplemented as a rotary actuator, such as, for example, the MG996Rservomotor available from Tower Pro of Taiwan, that generates circularto linear motion via a generally planar rotating disk 314 that ismechanically linked to the actuator 312. A rod 316 extending normal tothe plane of the disk 314 is attached at a point on the disk 314 that ispreferably at a radial distance from a central spindle 315 of at least50% of the radius of the disk 314, and more preferably at a radialdistance from the central spindle 315 of approximately 75% of the radiusof the disk 314.

The IR filter 302 is attached to the actuator 312 via an aperture 308located at a first end 304 of the IR filter 302. The aperture 308 andthe rod 316 are correspondingly configured, such that the rod 316 fitsthrough the aperture 308. The IR filter 302 is secured to the rod 316via a fastening arrangement, such a mechanical fastener. For example,the rod 316 may be implemented as a bolt having a shank portion and athreaded portion. In such an implementation, the bolt (rod 316) ispassed through the aperture 308 of the IR filter 302, and a nut havingcomplementary threading to the bolt is secured to the bolt to attach thefilter 302 to the actuator 312.

In operation, as the actuator 312 rotates the disk 314 about the centralspindle 315, the rotational movement of the disk 314 drives the IRfilter 302 and induces linear movement of the IR filter 302, therebymoving a second end 306 of the IR filter 302 into and out of the opticalpath 350 so as to block and unblock the lens 116.

According to certain embodiments, the IR filter assembly 300 includes aguiding arrangement 318 attached to the housing 117 of the imagingdevice 114 for guiding the IR filter 302 along a guide path 324. Theguiding arrangement 318 delimits the movement of the IR filter 302during movement in and out of the optical path 350. The guidingarrangement 318 preferably includes a pair of parallel guide rails thatdefine the guide path 324. In the drawings, the parallel guide rails aredepicted as a first guide rail 320 and a second guide rail 322, that arepositioned generally tangent to the lens 116 at diametrically opposedperipheral portions of the lens 116. In the preferred but non-limitingembodiments illustrated in FIGS. 14-16B, the guiding arrangement 318 andthe guide path 324 are positioned in front of the lens 116.

With particular reference to FIGS. 15A and 16A, there are shownschematic front and side views, respectively, of the IR filter assembly300 with the positioning mechanism 310 assuming a first state in whichthe IR filter 302 is positioned out of the optical path 350. As will bediscussed in subsequent sections of the present disclosure, thepositioning mechanism 310 assumes the first state when the system 10operates in calibration mode.

Looking now at FIGS. 15B and 16B, there are shown schematic front andside views, respectively, similar to FIGS. 15A and 16A, but with thepositioning mechanism 310 assuming a second state in which the IR filter302 is positioned in the optical path 350 so as to block the lens 116.As will be discussed in subsequent sections of the present disclosure,the positioning mechanism 310 assumes the second state when the system10 operates in operational mode.

FIGS. 15C and 15D show schematic front views illustrating thepositioning mechanism 310 assuming intermediate states between the firstand second states. Particularly, FIG. 15C shows the positioningmechanism 310 assuming an intermediate state in transition from thefirst state to the second state in which the IR filter 302 is intransition from out of the optical path 350 to into the optical path350. FIG. 15D shows the positioning mechanism 310 assuming anintermediate state in transition from the second state to the firststate in which the IR filter 302 is in transition from in the opticalpath 350 to out of the optical path 350. As the positioning mechanism310 moves between the first and second states, the IR filter 302 isguided along the guide path 324 into position to block the lens 116(FIG. 15C and then FIG. 15B), and then out of position to unblock thelens 116 (FIG. 15D and then FIG. 15A). The guide rails 320, 322 preventthe IR filter 302 from unwanted slipping into or out of the optical path350 during movement by the actuator 312.

With continued reference to FIGS. 15A-16B, refer now to FIG. 17, asimplified block diagram showing the connection between the IR filteringassembly 300 and the end unit 100 and the control subsystem 140. In anon-limiting implementation, the actuator 312 is linked to thecommunication and processing components of the end unit 100 such thatthe actuator 312 can be controlled by the control subsystem 140 via theend unit 100 over the network 150. In certain embodiments, some or allof the components of the IR filter assembly 300 are mechanicallyattached to, or integrated as part of, the end unit 100. In otherembodiments, the IR filter assembly 300 includes a dedicated receiverand processing unit for receiving commands from the control subsystem140 over the network 150 and relaying the received commands to theactuator 312.

When the system 10 operates in calibration mode, the control subsystem140 controls the positioning mechanism 310 to position the IR filter 302out of the optical path 350 such that the imaging device 114 captures afull spectral image of the scene (including the target 34), where theterm “full spectral image” generally refers to an image that conveysvisible and IR light image components of a scene. Generally speaking,the operation of the system 10 in calibration mode in embodimentsutilizing the IR filter assembly 300 is the same as the operation of thesystem 10 in calibration mode in embodiments without the IR filterassembly 300. As described for the embodiments corresponding to FIGS.1-13, when the system 10 operates in calibration mode the controlsubsystem 140 actuates the imaging device 114 to capture an image of thescene, which includes the target 34, and actuates one or more of theprocessing components of the system 10 (e.g., the image processingengine 134 of the processing subsystem 132, the processing unit 102) toprocess (i.e., analyze) the captured image in order to identify thetarget 34 in the scene and extract spatial information related to(associated with) the target 34. The spatial information related to thetarget includes the target size/dimensions (i.e., the horizontal andvertical dimensions of the target 34) and the horizontal and verticalposition of the target 34 within the scene, which together define thetarget coverage zone. The image processing engine 134 and/or theprocessing unit 102 may process the captured image by applying one ormore machine vision algorithms, which allow the processing components ofthe system 10 to define the target coverage zone from the extractedspatial information, enabling the processing components of the system 10to identify projectile strikes during operational mode by comparingsubsequently captured images against the extracted spatial information.The spatial information extraction and identification of the target 34in the scene may also be performed by imaging a bar code positioned nearthe target 34.

When the system 10 operates in operational mode, the control subsystem140 actuates the positioning mechanism 310 to position the IR filter 302into the optical path 350. In the illustrated embodiments, thepositioning of the IR filter 302 into the portion 352 of the opticalpath 350 entails placement of the IR filter 302 in front of the lens116, between the lens 116 and the scene, at a sufficient distance suchthat all of the light from the scene within the field of view 118necessarily passes through the IR filter 302 before impinging on thelens 116. In practice, the distance between the IR filter 302 and thelens 116 is on the order of several millimeters (e.g., 5-25 mm). Asdiscussed, the IR filter 302 blocks the visible light within the fieldof view 118 such that only the IR light within the field of view 118reaches the image sensor 115. This deployment of the IR filter 302 inthe optical path 350 effectively transforms the imaging device 114 intoan IR imaging device (since the image sensor 115 is sensitive toradiation in the IR region of the electromagnetic spectrum). Once the IRfilter 302 is positioned in the optical path 350, the control subsystem140 actuates the imaging device 114 to capture a series of images (IRimages) of the scene (target 34) and actuates one or more of theprocessing components of the system 10 (e.g., the image processingengine 134 of the processing subsystem 132, the processing unit 102) toprocess (i.e., analyze) the series of captured images.

The image captured based identification of projectile strikes inembodiments utilizing the IR filter assembly 300 is generally the sameas in embodiments without the IR filter assembly 300. The maindifference between the captured images in the present embodiments (usingthe IR filter assembly 300) and the captured images in the previouslydescribed embodiments (without the IR filter assembly 300) is that thecaptured images of the present embodiments are IR images—since only theIR light from the scene is successfully passed through the IR filter 302to the image sensor 115. As described for the embodiments correspondingto FIGS. 1-13, when the system 10 operates in operational mode the imageprocessing engine 134 and/or the processing unit 102 compare individualimages in the series of images with one or more other images in theseries of images to identify changes in the scene. These identifiedchanges are correlated with the target coverage zone, defined fromextracted spatial information during calibration mode, to identifyprojectile strikes on the target 34. For example, the image processingengine 134 and/or the processing unit 102 detect a projectile strike onthe target 34 in response to identifying a change in the portion of thescene that corresponds to the target coverage zone, whereby the changein the portion of the scene is identified via comparison between imagesin the series of images.

The control subsystem 140 is preferably configured to display the imageof the target 34 captured during calibration mode on a display devicecoupled to the control subsystem 140. In implementations in which thecontrol subsystem 140 is implemented as a management application 242executed on a mobile communication device 240 (FIG. 13), the image ofthe target 34 is displayed on the display area 244 of the display unitof the mobile communication device 240. In addition, the controlsubsystem 140 is preferably configured to overlay projectile strikeinformation extracted from the series of images captured by the imagingdevice 114 during operational mode. The projectile strike information isextracted from the series of images by the image processing engine 134and/or the processing unit 102, and is represented for display to theuser of the system 10 as demarcations, for example, dots, overlaid onthe image of the target 34.

It is noted that the IR filter assembly 300 can be deployed in variousconfigurations. For example, in certain non-limiting implementations theIR filter assembly 300 is deployed with the actuator 312 positionedbelow the imaging device 114 and with the guide rails 320, 322vertically oriented such that the IR filter 302 essentially moves in avertical fashion to block and unblock the lens 116. In suchimplementations, the majority of the motion of the IR filter 302 is inthe vertical direction (for example, as illustrated in FIGS. 16A and16B). In other implementations, the IR filter assembly 300 is deployedwith the actuator 312 adjacent to the imaging device 114 and with theguide rails 320, 322 horizontally oriented such that the IR filter 302essentially moves in a horizontal fashion to block and unblock the lens116. In such implementations, the majority of the motion of the IRfilter 302 is in the horizontal direction. In yet other implementations,the IR filter assembly 300 is deployed with the actuator 312 off-axisrelative to the vertical/horizontal directions of the imaging device 114and with the guide rails 320, 322 oriented at a corresponding anglerelative to the vertical/horizontal directions.

Although the embodiments of the IR positioning assembly described thusfar have pertained to deployment of an IR filter external to the imagingdevice 114 such that the IR filter is selectively positionable in aportion 352 of the optical path 350 between the scene and the lens 116(i.e., in front of the lens 116), other embodiments are possible inwhich the positioning mechanism 310 and the IR filter 302 are deployedinside of the imaging device 114 so as to enable positioning of the IRfilter 302 in and out of an optical path portion 354 between the imaginglens 116 and the image sensor 115. As should be apparent, in suchembodiments, the IR filter 302 is positionable in back of the lens 116and the guiding arrangement should also be attached to an internalportion of the housing of the imaging device 114 at the back of the lens116.

As alluded to above, according to certain embodiments the image sensor115 and the IR sensor 122 may be used in tandem, whereby the imagesensor 115 is used when the system 10 operates in calibration mode, andthe IR sensor 122 is used when the system 10 operates in operationalmode. FIG. 18 illustrates a generalized block diagram of the end unit100 according to such embodiments. Although preferably the two imagesensors 115, 122 are housed together in a single imaging device 114 (asshown in FIG. 18), the present embodiments include variations in whichthe image sensor 115 and the IR sensor 122 are housed in separateimaging devices.

In embodiments which employ using the image sensor 115 and the IR sensor122 in tandem, when the system 10 operates in calibration mode thecontrol subsystem 140 actuates the imaging device that houses the imagesensor 115 to capture an image of the scene (that includes the target34) using the image sensor 115. The captured image of the scene includesat least a visible light image, and may also include IR imageinformation if the image is a full spectral image (e.g., if the dichroicfilter of the imaging device has been removed or disabled). As discussedfor the previously described embodiments, the control subsystem 140 thenactuates one or more of the processing components of the system 10(e.g., the image processing engine 134, the processing unit 102) toprocess (i.e., analyze) the captured image in order to identify thetarget 34 in the scene and extract spatial information related to(associated with) the target 34.

When the system 10 operates in operational mode the control subsystem140 actuates the imaging device that houses the IR sensor 122 to capturea series of IR images of the scene (target 34) using the IR sensor 122.The control subsystem 140 then actuates one or more of the processingcomponents of the system 10 (e.g., the image processing engine 134, theprocessing unit 102) to process (i.e., analyze) the series of capturedIR images to detect projectile strikes on the target 34, as discussed inthe previously described embodiments.

The concept of IR filtering and/or switching between visible lightimaging and IR imaging described above in order to detect projectilestrikes on a target may also be applicable to detection of shooterevents at the shooter side (i.e., discharging of projectiles by ashooter firearm). Detection of shooter events may be of particular valuein joint firearm training (also referred to as “collaborative training”)environments, in which a plurality of shooters aims respective firearmsat a target. In such scenarios, it may be desirable to correlate firearmprojectile discharges with projectile strikes on a target, so as todetermine, for a given projectile strike on a target, the firearm (andhence shooter) that discharged the target-striking projectile.

Within the context of this document, the term “discharge” as used withrespect to discharging a firearm or discharging a projectile from afirearm, refers to the firing of the projectile from the firearm inresponse to actuating the firearm trigger mechanism. In situations inwhich the projectile is a live fire projectile such as a bullet or othertype of ammunition round, the act of discharging a firearm ordischarging a projectile from the firearm, as used within the context ofthe present disclosure and the appended claims, refers to the act ofexpelling the projectile from the barrel of the firearm during shootingin response to actuating the firearm trigger. In situations in which theprojectile is a beam of light (e.g., laser pulse), the act ofdischarging a firearm or discharging a projectile from the firearm, asused within the context of the present disclosure and the appendedclaims, refers to the act of emission of the light beam from a lightsource coupled to the firearm in response to actuating the light sourcevia a triggering mechanism.

Referring now to FIGS. 19-24, there is illustrated various aspects of ajoint firearm training system (referred to interchangeably as a “jointtraining system”) deployed in an environment that supports joint firearmtraining. Here a plurality of shooters, designated 402, 410, 418,operate associated firearms 404, 412, 420 to discharge projectiles 406,414, 422 with a goal of striking a target 426 deployed in a target area425. An end unit 100 is deployed to detect strikes of projectiles, inresponse to the firing of the firearms 404, 412, 420, on the target 426(as described above). The projectiles may be ballistic projectiles,i.e., live fire projectiles (i.e., bullets), or may be light-basedprojectile (such as the projectiles discharged by the firearm 20′).

The end unit 100 is deployed (i.e., positioned against the target 426)in accordance with the deployment methodologies described above withreference to FIGS. 1-18. In addition, the end unit 100 that is deployedagainst the target 426 may be configured to operate according to any ofthe embodiments described above with reference to FIGS. 1-18. Forexample, the end unit 100 may have a single imaging device that has avisible light image sensor that is coupled to an IR filter assembly 300to enable calibration of the end unit using visible light image capture,and projectile strike detection using IR image capture (as describedwith reference to FIGS. 14-15D). Alternatively, the end unit 100 mayhave an IR image sensor and a visible light image sensor to supportcalibration and projectile strike detection (as described with referenceto FIG. 18).

The target 426 may be a physical target or a virtual target, similar toas discussed in the embodiments described above with reference to FIGS.1-18.

It is noted that although three shooters are shown in this non-limitingexample, it should be apparent that the system may support a largernumber of shooters. Note that although FIG. 19 shows multiple shootersaiming respective firearms at a single common target (i.e., the target426), the joint firearm training methodologies of the present disclosureare also applicable to situations in which the target area 425 covers aplurality of spaced apart targets (arranged, for example, in an array,for example as illustrated in FIG. 10), and each individual shooter aimshis respective firearm at a respective dedicated target, or subsets(groups) of shooters aim respective firearms at a common target. Thejoint training methodologies described herein may also be applicable toenvironments in which a subset of shooters and targets are deployed at afirst geographic location, and a second subset of shooter and targetsare deployed at a second separate geographic location.

Continuing with the non-limiting example illustrated in FIG. 19, ashooter-side sensor arrangement 430 having at least one imaging deviceis deployed in front of the shooters 402, 410, 418 so as to cover acoverage area in which the shooters 402, 410, 418 are positioned. The atleast one imaging device is deployed to capture images of the shootersand their respective firearms so as to enable a processing system toprocess (analyze) the captured images to identify the shooters and/orfirearms, and to detect projectile discharges by the firearms. Theshooter/firearm identification is preferably performed by analyzing (bythe processing system) visible light images captured by an imagingdevice of the shooter-side sensor arrangement 430, whereas theprojectile discharge detection is performed by analyzing (by theprocessing system) IR images captured by an imaging device of theshooter-side sensor arrangement 430.

In a first non-limiting embodiment, the shooter-side sensor arrangement430 includes a single imaging device 432 that captures both visiblelight and IR images. The imaging device 432 is preferably implemented asa visible light imaging device, i.e., visible light camera, thatoperates on principles similar to the imaging device 114. An IR filterassembly 300′ is coupled to the imaging device 432 to enable bothvisible and IR image capture using the single imaging device 432. Theimaging device 432 has an image sensor 434 (i.e., detector) that issensitive to wavelengths in the visible light region of theelectromagnetic spectrum, and an optical arrangement having at least onelens 436 (including an imaging lens) which defines a field of view 468of a scene to be imaged by the imaging device 432. The lens 436 furtherdefines an optical path from the scene to the imaging device 432, and inparticular from the scene to the image sensor 434. As schematicallyillustrated in FIG. 20, the imaging device 432 is deployed such that theshooters 402, 410, 418 (and their associated firearms 404, 412, 420) arewithin the FOV 468 (i.e., the scene includes the shooters and theirassociated firearms). The shooter-side sensor arrangement 430 ispreferably deployed such that the target area 425 (and the target 426)are outside of the FOV of the imaging device 432.

It is noted that the structure and operation of the IR filter assembly300′ is identical to that of the IR filter assembly 300, and should beunderstood by analogy thereto. The same component numbering used toidentify the components of the IR filter assembly 300 is used toidentify the components of the IR filter assembly 300′, except that anapostrophe “'” is used to denote the components of the IR filterassembly 300′. Similar to the IR filter assembly 300, the IR filterassembly 300′ of the present embodiment has a positioning mechanism 310′that is operative to selectively move/position an IR filter 302′ in andout of a portion of the optical path from the scene to the imagingdevice 432, and more particularly from the scene to the image sensor434. It is noted that the deployment of the IR filter assembly 300′relative to the imaging device 432 is generally similar to that asdescribed above with respect to the deployment of the IR filter assembly300 relative to the imaging device 114, and therefore details of thedeployment will not be repeated here. One detail which will be repeatedhere pertains to the guiding arrangement 318′ of the IR filter assembly300′, which similar to the guiding arrangement 318, is attached to ahousing 437 of the imaging device 432. The controlled switching of theIR filter 302′ in and out of the optical path will be described ingreater detail in subsequent sections of the present disclosure withreference to FIGS. 22A-23B.

The imaging device 432 is associated with a processing system. Ingeneral, the processing system is configured to receive, from theimaging device 432, the images captured by the imaging device 432, andprocess the received images to: uniquely identify the shooters 402, 410,418 (and/or associated firearms 404, 412, 420), detect projectiledischarges by the firearms 404, 412, 420, and associate the detecteddischarged projectiles with the shooters operating the firearms thatdischarged the detected projectiles. The processing system is furtherconfigured to correlate the detection of discharged projectiles withdetections of projectile strikes on the target 426 by the end unit 100(where the projectile strike detection is performed by the end unit 100as described in detail above).

Many implementations of the processing system are contemplated herein.In one non-limiting implementation, a processing unit 456 is deployed aspart of the shooter-side sensor arrangement 430, and is electricallyassociated with the imaging device 432. The processing unit 456receives, from the imaging device 432, the images captured by theimaging device 432, and processes the received images to identify theshooters and detect projectile discharges. FIG. 21 shows a non-limitingexample of a block diagram of the processing unit 456. Here, theprocessing unit 456 includes a processor 458 coupled to an internal orexternal storage medium 460 such as a memory or the like, and a clock461. The external storage medium 460 may be implemented as an externalmemory device connected to the processing unit 456 via a data cable orother physical interface connection, or may be implemented as a networkstorage device or module, for example, hosted by a remote server (e.g.,the server 130). The clock 461 includes timing circuitry forsynchronizing the shooter-side sensor arrangement 430 and the end unit100, as will be discussed in subsequent sections of the presentdisclosure.

In other implementations, the imaging device 432 includes an embeddedprocessing unit that is part of the imaging device 432, and the embeddedprocessing unit performs the shooter identification and projectiledischarge detection. In other implementations, the shooter-side sensorarrangement 430 is linked to the network 150, and the images captured bythe imaging device 432 are provided to the processing subsystem 132(which is part of, or is hosted by, the server 130) via the network 150.Here, the processing subsystem 132, which is remotely located from theshooter-side sensor arrangement 430, performs the shooter identificationand projectile discharge detection based on images received from theimaging device 432. It is noted that the processing of the images may beshared between the processing unit 456 and the remote processingsubsystem 132.

The following paragraphs describe several exemplary methods foridentifying firearms and/or shooters as performed by the processingsystem according to embodiments of the present disclosure. As mentionedabove, the “processing system” may be any one of the processing unit456, embedded processing unit, processing subsystem 132, or acombination thereof. In certain embodiments, the processing systemidentifies the shooters and/or firearms by applying various machinelearning and/or computer vision algorithms and techniques to visiblelight images captured by the imaging device 432. In certain embodiments,one or more visual parameters in the visible light images associatedwith each of the shooters and/or firearms are evaluated.

In certain embodiments, the processing system is configured to analyzethe images captured by the imaging device 432 using facial recognitiontechniques to identify individual shooters. In such embodiments, each ofthe shooters may provide a baseline facial image (e.g., digital imagecaptured by a camera system) to the joint training system, which may bestored in a memory of the joint training system, for example the storagemedium 460 or the server 130 (which is linked to the shooter-side sensorarrangement 430 via the network 150). The processing system may extractlandmark facial features (e.g., nose, eyes, cheekbones, lips, etc.) fromthe baseline facial image. The processing system may then analyze theshape, position and size of the extracted facial features. In operation,the processing system identifies facial features in the images capturedby the imaging device 432 by searching through the captured images forimages with matching features to those extracted from the baselineimage.

In another embodiment, computer vision techniques are used to identifyshooters based on markers attached to the bodies of the shooters or thefirearms operated by the shooters. As shown in FIG. 19, a marker 408 isattached to a headpiece worn by the shooter 402, a marker 416 isattached to a headpiece worn by the shooter 410, and a marker 424 isattached to a headpiece worn by the shooter 418.

In a non-limiting implementation, the markers 408, 416, 424 arecolor-coded markers, with each shooter/firearm having a uniquelydecipherable color. In the non-limiting example deployment of the jointtraining system illustrated in FIG. 19 with three shooters, the shooter402 may have a red marker attached to his body or firearm 404, theshooter 410 may have a green marker attached to his body or firearm 412,and the shooter 418 may have a blue marker attached to his body orfirearm 420. The marker colors may be provided to the processing systemprior to operation of the joint training system. In operation, theprocessing system identifies the color-coded markers in the imagescaptured by the imaging device 432 which enables identification of theindividual shooters and/or firearms.

In another non-limiting implementation, the marker may be implemented asan information-bearing object, such as, for example, a bar code, thatcarries identification data. The bar code may store encoded informationthat includes the name and other identifiable characteristics of theshooter to which the bar code is attached. In operation, the processingsystem searches for bar codes in the images captured by the imagingdevice 432, and upon finding such a bar code, decodes the informationstored in the bar code, thereby identifying the shooter (or firearm) towhich the bar code is attached.

In another embodiment, the processing system may be configured toidentify individual shooters according to geographic position of eachshooter within the FOV 468 of the imaging device 432. In suchembodiments, the FOV 468 of the imaging device 432 may be sub-dividedinto non-overlapping sub-regions (i.e., sub-coverage areas), with eachshooter positioned in a different sub-region. FIG. 20 shows a schematicrepresentation of the sub-division of the FOV 468 into threesub-regions, namely a first sub-region 470, a second sub-region 472, anda third sub-region 474. The shooter 402 is positioned in the firstsub-region 470, the shooter 410 is positioned in the second sub-region472, and the shooter 418 is positioned in the third sub-region 474. Thesub-division of the FOV 468 may be pre-determined (i.e., prior tooperation of the joint training system to perform the joint trainingdisclosed herein) Likewise, the requisite position of each of theshooters, in the respective sub-regions of the FOV may be pre-assignedand provided to the processing system. In operation, the processingsystem analyzes the images captured by the imaging device 432 toidentify the shooters according to the pre-defined position in the FOVsub-regions 470, 472, 474.

A control system is associated with (linked to) the shooter-side sensorarrangement 430 (and in particular the imaging device 430 and the IRfilter assembly 300′) in order to allow the imaging device 432 to switchbetween visible light and IR image capture. In order to capture thevisible light images (which are to be analyzed by the processing systemto identify the shooters and/or firearms as described above), thecontrol system actuates the positioning mechanism 310′ so as to move theIR filter 302′ to a position in which the IR filter 302′ is positionedout of the optical path from the scene to the imaging device 432. Thecontrol system then actuates the imaging device 432 to capture images ofthe scene (which are visible light images of the shooter and/orfirearms), which are then processed by the processing system to identifythe shooters and/or firearms. In order to capture IR images (which areto be analyzed by the processing system to detect projectile dischargeevents), the control system actuates the positioning mechanism 310′ soas to move the IR filter 302′ to a position in which the IR filter 302′is positioned in the optical path from the scene to the imaging device432.

It is noted that the control system may actuate the imaging device 432to capture the IR images and the visible light images during respectiveimage capture time intervals, which coincide with the time intervalsduring which the control system actuates the positioning mechanism 310′to position the IR filter 302 in the optical path and out of the opticalpath, respectively. In one non-limiting example, there is a single IRimage capture interval and a single visible light image captureinterval, such that the imaging device 432 captures two temporallynon-overlapping sets of images in sequence, for example first capturinga set/series of IR images and then capturing a set/series of visiblelight images (or vice versa). In another non-limiting example, there aremultiple IR image capture intervals interleaved with multiple visiblelight image capture intervals. In such an example, the control systemessentially actuates the imaging device 432 to capture images whileactuating the positioning mechanism 310′ to switch the IR filter 302 inand out of the optical path. In this way, the imaging device 432captures images while as the IR filter 302 switches back and forth,resulting in the capture of interleaved sets of IR and visible lightimages. In all cases, the control system preferably provides informationpertaining to the type of image (IR or visible light) that was capturedby the imaging device 432 to the processing system, so that theprocessing system can process the visible light and IR images inaccordance with the different processing techniques described above soas to be able to perform the shooter/firearm identification andprojectile discharge detection.

In certain non-limiting implementations, the control subsystem 140 mayprovide the control functionality for actuating the positioningmechanism 310′ to switch the IR filter 302′ into and out of the opticalpath. In other non-limiting implementations, the control system and theprocessing system are implemented using a single processing system so asto provide both control and processing functionality using a singleprocessing system. In other words, in such implementations, theprocessing system also provides the control functionality for actuatingthe positioning mechanism 310′ switch the IR filter 302 in and out ofthe optical path.

FIGS. 22A-23B show schematic front and side views of the IR filterassembly 300′ deployed with the imaging device 432. In FIGS. 22A and22A, the positioning mechanism 310′ is shown assuming a first state inwhich the IR filter 302′ is positioned out of the optical path(generally designated 450). In FIGS. 22B and 23B, the positioningmechanism 310′ is shown assuming a second state in which the IR filter302′ is positioned in the optical path 450. The optical path 450 fromthe scene to the imaging device 432 is generally defined herein as theregion of space through which light from the scene can traverse directlyto and through the imaging device 432 so as to be imaged by the lens 436onto the image sensor 434. The optical path 450 overlaps entirely withthe field of view 468 defined by the lens 436, and includes two opticalportions. A first optical path portion (generally designated 452)between the scene and the lens 436, and a second optical path portion(generally designated 454) between the lens 436 and the image sensor434. In the preferred but non-limiting implementations illustrated inFIGS. 22A-23B, the IR filter 302 is positionable a short distance infront of the lens 436, and between the lens 436 and the scene, i.e., theportion of the optical path 450 is the optical path portion 452 betweenthe scene and the lens 436.

It is noted that the size of the shooters 402, 410, 418, positioned inthe optical path 450, are not shown to scale in the schematicrepresentations shown in FIGS. 23A and 23B.

When the IR filter 302′ is positioned in the optical path 450, all ofthe light from the scene within the field of view 468 passes through theIR filter 302′, such that the visible light within the field of view 468is blocked by the IR filter 302′ and only the IR light within the fieldof view 468 reaches the image sensor 434. Conversely, when the IR filter302′ is positioned out of the optical path 450, none of the light fromthe scene passes through the IR filter 302′ such that all of the light(both visible and IR) from the scene within the field of view 468reaches the image sensor 434. Since the image sensor 434 is preferablyimplemented as an image sensor that is sensitive to wavelengths in thevisible light region of the electromagnetic spectrum, only visible lightis imaged by the imaging device 432 when the IR filter 302′ ispositioned out of the optical path 450.

Turning now to the detection of projectile discharges, these “projectiledischarge events” are typically in the form of exit blasts from thefirearm barrel or light-pulses output from light-emitters (e.g., as inthe firearm 20′). These projectile discharge events are most easilydetectable when utilizing IR imaging to capture images of the scene. Inorder to capture the IR images, the control system actuates thepositioning mechanism 310′ to assume the second state such that the IRfilter 302′ is positioned in the optical path 450.

The imaging device 432, now operating as an IR imaging device, capturesa series of IR images, and the IR images are analyzed (processed) by theprocessing system so as to detect projectile discharges by the firearms.The processing system is configured to receive, from the imaging device432, the series of IR images captured by the imaging device 432. Theprocessing system processes (analyzes) the received series of IR imagesto detect projectile discharge events (referred to interchangeably as“projectile discharges”) from each of the firearms of the shooters inthe FOV 468. Each detected projectile discharge is made in response to ashooter firing his/her associated firearm. For example, in anon-limiting implementation in which the imaging device 432 is deployedto capture images of all three of the shooters 402, 410, 418, theprocessing system is configured to detect the discharging of theprojectiles 406, 414, 422, in response to the shooters 402, 410, 418firing the respective firearms 404, 412, 420, thereby yielding threeprojectile discharge events.

The processing system may analyze the received shooter-side IR images invarious ways. In a preferred but non-limiting exemplary implementation,the processing system implements machine/computer vision techniques toidentify flashes, corresponding to projectile discharges, from thebarrel of the firearm. In another non-limiting exemplary implementation,the processing system may detect projectile discharges via thermographictechniques, for example by detecting the heat signature of theprojectile as it leaves the barrel of the firearm.

In another non-limiting implementation, which may be alternative to orin combination with the machine/computer vision techniques orthermographic implementation, individual images in the series of IRimages are compared with one or more other images in the series ofimages to identify changes between images, in order to identify theflashes coming from the barrel of the firearm corresponding toprojectile discharges.

Preferably, the processing system links an identified projectiledischarge with the firearm that discharged the projectile, based theidentification of the firearms and/or shooters described above.

The linking may be performed, for example, by determining which of theidentified firearms and/or shooters is closest in proximity to which ofthe identified projectile discharges. The proximity may be evaluated ona per pixel level, for example by determining the differences in pixellocation between IR image pixels indicative of a projectile dischargeand visible image pixels indicative of an identified firearm and/orshooter.

In preferred embodiments, the processing system is further configured tocorrelate the detected projectile discharges (which are linked toindividual shooters) with projectile strikes on the target that aredetected by the end unit 100 (which can be considered as a “target-sidesensor arrangement”). It is assumed that by detecting projectile strikeson the target 426, the end unit 100 is already calibrated (which can beaccomplished using any of the calibration methodologies described abovewith reference to FIGS. 1-18, which will not be repeated here). In orderto perform the correlation, the processing system preferablysynchronizes the end unit 100 and the shooter-side sensor arrangement430. The synchronization is effectuated, in certain non-limitingimplementations, by direct linking of the processing system to the endunit 100 and the shooter-side sensor arrangement 430. In anothernon-limiting implementation, the synchronization is effectuated byutilizing timing circuitry deployed at the end unit 100 and at theshooter-side sensor arrangement 430. The timing circuitry of the endunit 100 and the shooter-side sensor arrangement 430 are represented asclocks 161 and 461 in FIGS. 3 and 21, respectively. It is noted thatalthough the clock 461 is shown as being a part of the processing unit456, this is for simplicity of illustration only. Other implementationsare contemplated herein in which the clock 461 (or any other timingcontrol circuitry) is a part of, or is linked to, the shooter-sidesensor arrangement 430.

The clocks 161 and 461 may provide temporal information (e.g., timestampinformation), to the processing system, for each of the images capturedby imaging devices 114 and 432. In other embodiments, the processingsystem may apply timestamps to the data received from the end unit 100and the shooter-side sensor arrangement 430, thereby providing temporalinformation for the detection events (i.e., the projectile dischargeevents and the projectile strike events).

The shooter-side sensor arrangement 430 may also be functionallyassociated with a distance measuring unit 444 that is configured tomeasure (i.e., estimate) the distance between the shooter-side sensorarrangement 430 and each of the shooters 402, 410, 420. The distancemeasuring unit 444 may be implemented, for example, as a laserrangefinder that emits laser pulses for reflection off of a target(i.e., the shooters) and calculates distance based on the timedifference between the pulse emission and receipt of the reflectedpulse.

In certain embodiments, the distance measuring unit 444 may be absentfrom the shooter-side sensor arrangement 130, and the distance betweenthe shooter-side sensor arrangement 430 and each of the shooters 402,410, 420 may be calculated using principles of triangulation (i.e.,stereoscopic imaging) based on images captured by two shooter-sideimaging device 432 that are synchronized with each other. Alternatively,the imaging device 432 may be implemented as part of a stereo visioncamera system, such as the Karmin2 stereo vision camera available fromSODA VISION, that can be used to measure the distance between theshooter-side sensor arrangement 430 and each of the shooters 402, 410,420.

The end unit 100 may also have an associated distance measuring unit 144(which may be electrically linked to the end unit 100 or may be embeddedwithin the end unit 100), that is configured to measure the distancebetween the end unit 100 and the target area 425. The distance measuringunit 144 may be implemented, for example, as a laser rangefinder.Instead of estimating distance using a distance measuring unit 144, thedistance between the end unit 100 and the target area 425 may becalculated (i.e., estimated) by applying image processing techniques,performed by the processing system, to images (captured by the imagingdevice 114) of a visual marker attached to the target area 425. Thevisual marker may be implemented, for example, as a visual mark of apredefined size. The number of pixels dedicated to the portion of thecaptured image that includes the visual mark can be used as anindication of the distance between the end unit 100 and the target area425. For example, if the end unit 100 is positioned relatively close tothe visual mark, a relatively large number of pixels will be dedicatedto the visual mark portion of the captured image. Similarly, if the endunit 100 is positioned relatively far from the visual mark, a relativelysmall number of pixels will be dedicated to the visual mark portion ofthe captured image. As a result, a mapping between the pixel density ofportions of the captured image and the distance to the object beingimaged can be generated by the processing system, based on the visualmark size.

Note that distance measuring units may not be required to determine theabove-described distances. In certain embodiments, an operator of thejoint training system, which may be, for example, a manager of theshooting range in which the joint training system is deployed, or one ormore of the shooters 402, 410, 418, may manually input theaforementioned distances to the processing system. In such embodiments,manual input to the processing system may be effectuated via userinterface (e.g., a graphical user interface) executed by a computerprocessor on a computer system linked to the processing system. In suchembodiments, the processing system may be deployed as part of thecomputer system that executes the user interface.

In certain embodiments, the shooter-side sensor arrangement 430 and theend unit 100 are approximately collocated. The two distances (i.e.,between the shooter-side sensor arrangement 430 and the shooters, andbetween the target-side system (end unit 100) and the target area 425)are summed by the processing system to calculate (i.e., estimate) thedistance between the target area 425 and shooters 402, 410, 418. Thetypical distance between the shooter-side sensor arrangement 430 and theshooters 402, 410, 418 is preferably in the range of 6-8.5 meters, andthe distance between the end unit 100 and the target area 425 ispreferably in the range of 0.8-1.5 meters. Accordingly, in anon-limiting deployment of the joint training system, the distancebetween the shooters 402, 410, 418 and the target area 425 is in therange of 6.8-10 meters.

In other embodiments, the sensor arrangement 430 and the end unit 100are spaced apart from each other at a pre-defined distance. Such spacingmay support long-range shooting capabilities, in which the distancebetween the shooters 402, 410, 418 and the target area 425 may begreater than 10 meters (for example several tens of meters and up twoseveral hundred meters). In such an embodiment, the distance between theshooter-side sensor arrangement 430 and the shooters, between the endunit 100 and the target area 425, and the pre-defined distance betweenthe sensor arrangement 430 and the end unit 100 are summed by theprocessing system to calculate the distance between the target area 425and shooters 402, 410, 418.

Based on the calculated distance between the target area 425 andshooters 402, 410, 418, and the average speed of a dischargedprojectile, the processing system may calculate an expected time offlight (ToF), defined as the amount of time a discharged projectile willtake to strike the target area 425, for each firearm. The processingsystem may store the expected ToFs for each firearm in a memory (e.g.,the storage medium 460) or in a database as a data record with header orfile information indicating to which firearm (i.e., shooter) eachexpected ToF corresponds.

It is noted that the range between the object (e.g., shooters or target)to be imaged and the sensor arrangement 430 and end unit 100 may beincreased in various ways. For example, higher resolution image sensors,or image sensors with larger optics (e.g., lenses) and decreased FOV,may be used to increase the range. Alternatively, multiple shooter-sideimaging devices 432 with non-overlapping FOVs may be deployed toincrease the operational range between the shooters and the shooter-sidesensor arrangement 430.

In one non-limiting operational example, for each detected projectilestrike, the processing system evaluates the temporal information (i.e.,timestamp) associated with the projectile strike. The processing systemalso evaluates the temporal information associated with recentlydetected projectile discharges. The processing system then compares thetemporal information associated with the projectile strike with thetemporal information associated with recently detected projectiledischarges. The comparison may be performed, for example, by taking thepairwise differences between the temporal information associated withrecently detected projectile discharges and the temporal informationassociated with the projectile strike to form estimated ToFs. Theestimated ToFs are then compared with the expected ToFs to identify aclosest match between estimated ToFs and expected ToFs. The comparisonmay be performed by taking the pairwise differences between theestimated ToFs and the expected ToFs, and then identifying the estimatedToF and expected ToF pair that yields the minimum (i.e., smallest)difference.

Since the processing system provides synchronization between the eventsdetected in response to the data received from the sensor arrangement430 and the end unit 100, which in certain embodiments is provided viasynchronization of the clocks 161, 461, the processing system is able toperform the ToF calculations with relatively high accuracy, preferablyto within several micro seconds. Furthermore, by identifying theestimated ToF and expected ToF pair, the processing system is able toretrieve the stored information indicative of to which firearm (i.e.,shooter) is associated with the expected ToF, thereby attributing thedetected projectile strike to the shooter operating the firearmassociated with the expected ToF of the identified estimated ToF andexpected ToF pair. As such, the processing system is able to identify,for each detected projectile strike on the target area 425, thecorrespondingly fired firearm that caused the detected projectilestrike.

The processing system may also be configured to provide target missinformation for projectile discharges that failed to hit the target 426or the target area 425. To do so, the processing system may evaluatetemporal information associated with each detected projectile discharge.The processing system also evaluates the temporal information associatedwith recently detected projectile strikes. The processing system thencompares the temporal information associated with the projectiledischarge with the temporal information associated with recentlydetected projectile strikes. The comparison may be performed, forexample, by taking the differences between the temporal information,similar to as described above, to form estimated ToFs. Pairwisedifferences between the estimated ToFs and the expected ToFs may then beperformed. The estimated ToF and expected ToF pair that yields theminimum difference but is greater than a threshold value is attributedto the firearm (i.e., shooter) associated with the expected ToF as atarget miss.

The embodiments described above have thus pertained to capturing imagesof the shooters and firearms using a single imaging device 432 inoperation with an IR filter assembly 300′. Such embodiments provide acost-effective solution which enables visible light image capture toidentify the shooters, and IR image capture to detect projectiledischarge events. However, other embodiments are possible in which IRimage capture is performed using a dedicated IR image sensor. In suchembodiments, no IR filter assembly is deployed.

FIG. 24 illustrates a block diagram of the imaging device 432 accordingto such an embodiment, in which the imaging device 432 includes an IRimage sensor 435 in addition to the visible light image sensor 434. Insuch embodiments, the IR image sensor is sensitive to light in the IRregion of the electromagnetic spectrum. Preferably, the image sensors434 and 435 are boresighted such that they have a common FOV 468 (i.e.,light from the same scene reaches both sensors 434, 435). Althoughpreferably the two image sensors 434, 435 are housed together in asingle imaging device 432, the present embodiments include variations inwhich the image sensor 435 is housed in a separate imaging device fromthe imaging device 432.

In the present embodiments, the image sensors 434, 435 are used intandem in order to identify the shooters and/or firearms and to detectprojectile discharge events. In particular, visible light imagescaptured by the visible light sensor 434 are processed by the processingsystem to identify shooters and/or firearms (as described above). The IRimages captured by the IR image sensor 435 are processed by theprocessing system (similar to the IR images captured when the IR filter302 is deployed in the optical path as in FIGS. 22B and 23B) in order todetect the projectile discharge events.

The control system may controllably switch the imaging sensors 434, 435on to capture visible and IR images. For example, when the jointtraining system operates in one operating mode, the image sensor 434 maybe switched on to capture a series of visible light images of theshooters and/or firearms. When the joint training system operates inanother operating mode, the IR image sensor 435 may be switched on tocapture a series of IR images of the shooters and/or firearms so as tocapture ballistic flashes or pulsed-light flashes from the firearms.

Although embodiments have been described for multiple shooters,correlation of projectile discharge with projectile strikes may beapplicable to single shooter-single target scenarios. It is furthernoted that although the embodiments of the joint training systemdescribed above have pertained to a single shooter-side sensorarrangement having a switchable IR filter associated with a singleimaging device or having an IR image sensor and a visible light imagesensor, other embodiments are possible in which two or more suchshooter-side sensor arrangements are deployed so as to provide a degreeof image capture redundancy. For example, two shooter-side sensorarrangements 430 may be deployed, each having a single imaging device432 coupled to an IR filter assembly 300′, that is actuatable by thecontrol system.

It is noted that although a shooter-side sensor arrangement employingvisible light imaging and IR imaging is of particular value when used injoint shooter training scenarios in which multiple shooters dischargefirearms at one or more targets, such a shooter-side sensor arrangementcan equally be applicable to single shooter environments, where visiblelight images are used by the processing system to identify the shooterand IR images are used by the processing system to identify projectiledischarges. The processing system can correlate the identified/detectedprojectile discharges with detected projectile strikes on the targetusing the images captured by the end unit, as described above.

Implementation of the method and/or system of embodiments of theinvention can involve performing or completing selected tasks manually,automatically, or a combination thereof. Moreover, according to actualinstrumentation and equipment of embodiments of the method and/or systemof the invention, several selected tasks could be implemented byhardware, by software or by firmware or by a combination thereof usingan operating system.

For example, hardware for performing selected tasks according toembodiments of the invention could be implemented as a chip or acircuit. As software, selected tasks according to embodiments of theinvention could be implemented as a plurality of software instructionsbeing executed by a computer using any suitable operating system. Asdiscussed above, the data management application 242 may be implementedas a plurality of software instructions or computer readable programcode executed on one or more processors of a mobile communicationdevice. As such, in an exemplary embodiment of the invention, one ormore tasks according to exemplary embodiments of method and/or system asdescribed herein are performed by a data processor, such as a computingplatform for executing a plurality of instructions. Optionally, the dataprocessor includes a volatile memory for storing instructions and/ordata and/or a non-volatile storage, for example, non-transitory storagemedia such as a magnetic hard-disk and/or removable media, for storinginstructions and/or data. Optionally, a network connection is providedas well. A display and/or a user input device such as a keyboard ormouse are optionally provided as well.

For example, any combination of one or more non-transitory computerreadable (storage) medium(s) may be utilized in accordance with theabove-listed embodiments of the present invention. The non-transitorycomputer readable (storage) medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

The block diagrams in the drawings illustrate the architecture,functionality, and operation of possible implementations of systems,devices, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

As used herein, the singular form, “a”, “an” and “the” include pluralreferences unless the context clearly dictates otherwise.

The word “exemplary” is used herein to mean “serving as an example,instance or illustration”. Any embodiment described as “exemplary” isnot necessarily to be construed as preferred or advantageous over otherembodiments and/or to exclude the incorporation of features from otherembodiments.

It is appreciated that certain features of the invention, which are, forclarity, described in the context of separate embodiments, may also beprovided in combination in a single embodiment. Conversely, variousfeatures of the invention, which are, for brevity, described in thecontext of a single embodiment, may also be provided separately or inany suitable subcombination or as suitable in any other describedembodiment of the invention. Certain features described in the contextof various embodiments are not to be considered essential features ofthose embodiments, unless the embodiment is inoperative without thoseelements.

The processes (methods) and systems, including components thereof,herein have been described with exemplary reference to specific hardwareand software. The processes (methods) have been described as exemplary,whereby specific steps and their order can be omitted and/or changed bypersons of ordinary skill in the art to reduce these embodiments topractice without undue experimentation. The processes (methods) andsystems have been described in a manner sufficient to enable persons ofordinary skill in the art to readily adapt other hardware and softwareas may be needed to reduce any of the embodiments to practice withoutundue experimentation and using conventional techniques.

Although the invention has been described in conjunction with specificembodiments thereof, it is evident that many alternatives, modificationsand variations will be apparent to those skilled in the art.Accordingly, it is intended to embrace all such alternatives,modifications and variations that fall within the spirit and broad scopeof the appended claims.

What is claimed is:
 1. A firearm training system, comprising: an imagingdevice deployed to capture images of a scene, the scene including atleast one shooter, each shooter of the at least one shooter operating anassociated firearm to discharge one or more projectile; an infraredfilter; a positioning mechanism operatively coupled to the infraredfilter, the positioning mechanism configured to position the infraredfilter in and out of a path between the imaging device and the scene; acontrol system operatively coupled to the positioning mechanism andconfigured to: actuate the positioning mechanism to position theinfrared filter in and out of the path, and actuate the imaging deviceto capture images of the scene when the infrared filter is positioned inand out of the path; and a processing system configured to: processimages of the scene captured when the infrared filter is positioned inthe path to detect projectile discharges in response to each shooter ofthe at least one shooter firing the associated firearm, and processimages of the scene captured when the infrared filter is positioned outof the path to identify, for each detected projectile discharge, ashooter of the at least one shooter that is associated with the detectedprojectile discharge.
 2. The firearm training system of claim 1, whereinthe at least one shooter includes a plurality of shooters, and whereineach shooter operates the associated firearm with a goal to strike atarget with the discharged projectile, the firearm training systemfurther comprising: an end unit comprising an imaging device deployedfor capturing images of the target, and wherein the processing system isfurther configured to: process images of the target captured by imagingdevice of the end unit to detect projectile strikes on the target, andcorrelate the detected projectile strikes on the target with thedetected projectile discharges to identify, for each detected projectilestrike on the target, a correspondingly fired firearm associated withthe identified shooter.
 3. The firearm training system of claim 2,wherein the target is a physical target.
 4. The firearm training systemof claim 2, wherein the target is a virtual target.
 5. The firearmtraining system of claim 1, wherein the positioning mechanism includes amechanical actuator in mechanical driving relationship with the infraredfilter.
 6. The firearm training system of claim 1, wherein thepositioning mechanism generates circular-to-linear motion for moving theinfrared filter in and out of the path from the scene to the imagingdevice.
 7. The firearm training system of claim 1, wherein the imagingdevice includes an image sensor and at least one lens defining anoptical path from the scene to the image sensor.
 8. The firearm trainingsystem of claim 1, further comprising: a guiding arrangement inoperative cooperation with the infrared filter and defining a guide pathalong which the infrared filter is configured to move, such that theinfrared filter is guided along the guide path and passes in front ofthe at least one lens so as to be positioned in the optical path whenthe positioning mechanism is actuated by the control system.
 9. Thefirearm training system of claim 1, wherein the projectiles are liveammunition projectiles.
 10. The firearm training system of claim 1,wherein the projectiles are light beams emitted by a light sourceemanating from the firearm.
 11. The firearm training system of claim 1,wherein the control system and the processing system are implementedusing a single processing system.
 12. The firearm training system ofclaim 1, wherein the processing system is deployed as part of a serverremotely located from the imaging device and in communication with theimaging device via a network.
 13. A firearm training system, comprising:a shooter-side sensor arrangement including: a first image sensordeployed for capturing infrared images of a scene, the scene includingat least one shooter, each shooter of the at least one shooter operatingan associated firearm to discharge one or more projectile, and a secondimage sensor deployed for capturing visible light images of the scene;and a processing system configured to: process infrared images of thescene captured by the first image sensor to detect projectile dischargesin response to each shooter of the at least one shooter firing theassociated firearm, and process visible light images of the scenecaptured by the second image sensor to identify, for each detectedprojectile discharge, a shooter of the at least one shooter that isassociated with the detected projectile discharge.
 14. The firearmtraining system of claim 13, wherein the at least one shooter includes aplurality of shooters, and wherein each shooter operates the associatedfirearm with a goal to strike a target with the discharged projectile,the firearm training system further comprising: an end unit comprisingan imaging device deployed for capturing images of the target, andwherein the processing system is further configured to: process imagesof the target captured by imaging device of the end unit to detectprojectile strikes on the target, and correlate the detected projectilestrikes on the target with the detected projectile discharges toidentify, for each detected projectile strike on the target, acorrespondingly fired firearm associated with the identified shooter.15. The firearm training system of claim 14, wherein the target is aphysical target.
 16. The firearm training system of claim 14, whereinthe target is a virtual target.
 17. A firearm training method,comprising: capturing, by at least one image sensor, visible lightimages and infrared images of a scene that includes at least oneshooter, each shooter of the at least one shooter operating anassociated firearm to discharge one or more projectile; and analyzing,by at least one processor, the captured infrared images to detectprojectile discharges in response to each shooter of the at least oneshooter firing the associated firearm, and analyzing, by the at leastone processor, the captured visible light images to identify, for eachdetected projectile discharge, a shooter of the at least one shooterthat is associated with the detected projectile discharge.
 18. Thefirearm training method of claim 17, wherein the at least one imagesensor includes exactly one image sensor, and wherein infrared imagesare captured by the image sensor when an infrared filter is deployed apath between the image sensor and the scene, and wherein the visiblelight images are captured by the image sensor when the infrared filteris positioned out of the path between the image sensor and the scene.19. The firearm training method of claim 17, wherein the at least oneimage sensor includes: an infrared image sensor deployed for capturingthe infrared images of the scene, and a visible light image sensordeployed for capturing the visible light images of the scene.
 20. Thefirearm training method of claim 17, wherein the at least one shooterincludes a plurality of shooters, and wherein each shooter operates theassociated firearm with a goal to strike a target with the dischargedprojectile, the firearm training method further comprising: capturing,by an imaging device, images of the target; analyzing, by the at leastone processor, images of the target captured by imaging device to detectprojectile strikes on the target; and correlating the detectedprojectile strikes on the target with the detected projectile dischargesto identify, for each detected projectile strike on the target, acorrespondingly fired firearm associated with the identified shooter.